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(57) An apparatus for retrieving moving image ac- 
cording to the present disclosure comprises an extract- 
ing unit 1 02 for extracting information 1 03 of the feature 
quantity of moving image from fixed moving image data, 
an extracting unit 104 for extracting information 105 of 
the unique identification of moving image to identify the 
moving image according to the information 103 of the 
feature quantity of the moving image, and a retrieving 
unit 109 for retrieving and identifying the fixed moving 
image from object moving image for retrieving by com- 
paring the information 1 05 of the unique identification of 
the moving image with information 108 of the unique 
identification of moving image extracted beforehand 
from the object moving image. Therefore, the informa- 
tion of the unique identification of the moving image for 
retrieving the moving image which depends on the con- 
tent of the moving image can be extracted, consequent- 
ly, the identical moving image can be retrieved and iden- 
tified from the moving image by using the information of 
the unique identification of the moving image at high 
speed. 
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(57) An apparatus for retrieving moving image 
according to the present disclosure comprises an 
extracting unit 102 for extracting information 103 of the 
feature quantity of moving image from fixed moving 
image data, an extracting unit 104 for extracting infor- 
mation 105 of the unique identification of moving image 
to identify the moving image according to the informa- 
tion 1 03 of the feature quantity of the moving image, and 
a retrieving unit 109 for retrieving and identifying the 
fixed moving image from object moving image for 
retrieving by comparing the information 105 of the 
unique identification of the moving image with informa- 
tion 108 of the unique identification of moving image 
extracted beforehand from the object moving image. 
Therefore, the information of the unique identification of 
the moving image for retrieving the moving image which 
depends on the content of the moving image can be 
extracted, consequently, the identical moving image can 
be retrieved and identified from the moving image by 
using the information of the unique identification of the 
moving image at high speed. 
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Description 

FIELD OF THE INVENTION 

5 [0001] This invention relates to a method and an apparatus for retrieving moving image, which retrieve and identify 
fixed moving image from some moving image. And more particularly to, a method and an apparatus for retrieving mov- 
ing image, in which the information of the unique identification of moving image which depends on the content of the 
moving image is extracted and registered, the registered moving image is retrieved and identified earlier than video rate 
from moving image such as moving image archives sent in broadcasting and the like. 

10 

BACKGROUND OF THE INVENTION 

[0002] It is well known that there are various technologies on the international standard encoding system such as 
MPEG (Motion Picture Experts Group) as the information compressing technology for storing and transmitting moving 
15 image. It becomes possible that moving image is sufficiently handled even in the PC (Personal Computer), if the 
encoded moving image compressed by this international standard encoding system is used. The technology for moni- 
toring the specific moving image by using the PC, for example monitoring a television commercial (identifying moving 
image), is ever studied and is developed. 

[0003] As the conventional method for retrieving moving image, there are several methods for retrieving moving 
20 image, for example a method using the keyword retrieval, a method using the visual discrimination by the visual obser- 
vation of the user, and a method using the number of continuous frames on the shot of moving image disclosed in Jap- 
anese Patent Publication No. JP7-46517. 

[0004] According to the method using the keyword retrieval, moving image is given a classification code, a title, the 
content of the moving image, various relation information and so on as the information of keyword index, then the mov- 
25 ing image is retrieved and identified by retrieving this keyword. 

[0005] And, according to the method using the visual discrimination by the visual observation of the user, the user 
memorizes moving image that the user intends to retrieve and to identify then the moving image is retrieved and iden- 
tified by the user inspecting visually all moving images. 

[0006] And, according to the apparatus disclosed in Japanese Patent Publication No. JP7-46517, the number of 
30 continuous frames on each shot is searched between cut points by the means for detecting cut point, then moving 

image is retrieved by using the column of the number of continuous frames as index information. 

[0007] And, in the real time discrimination method of image scene based on the time series coding of cutting (Naga- 

saka others, April, 1996 Inst, of Electronics, Inf. and Commun. Engineering paper magazine D-ll Vol.J79-D-ll No.4 pp. 

531 -537), the frame image of a cut point is extracted, and average color information is calculated by dividing the frame 
35 into 2x2 and encoding, and the encoded information is arranged in the time series and applied to the discrimination 

(retrieval) of image scene. 

[0008] And, there is an apparatus for retrieving the cut point of moving image disclosed in Japanese Patent Publi- 
cation No. JP7-236153 as an apparatus for retrieving moving image by detecting the cut point of the moving image. In 
this apparatus for retrieving the cut point of moving image disclosed in Japanese Patent Publication No. JP7-236153, 
40 the time variation of luminance difference between frames of the reduction image which is made from prepared reduc- 
tion image and the mean value component data of the information of compressed and encoded moving image and the 
time variation of color difference histogram correlation value are used in order to obtain a cut point at high speed from 
moving image. 

[0009] And, there is an apparatus for processing moving image disclosed in Japanese Patent Publication No. JP7- 
45 1 52779 as an apparatus for detecting the characteristic frame of moving image. In this apparatus for processing moving 
image disclosed in Japanese Patent Publication No. JP7-1 52779, the low frequency part of DCT (Discrete Cosine 
Transform) component is used in order to detect characteristic frame that is used for as index information (retrieving 
information), for example a cut point frame, a motionless frame, a character frame, a specific color frame, etc., from 
compressed moving image data including the DCT operation. 
50 [0010] In the conventional methods and apparatuses for retrieving moving image, however, there are disadvan- 
tages as follows. 

[001 1 ] That is to say, in the conventional method for retrieving moving image by the keyword retrieval, there is a dis- 
advantage that the considerable cost for registering keyword index information is taken, because to beforehand register 
the keyword index information is necessary. And more, there is a disadvantage with the case in which keyword index 
55 information differs even in the case of retrieving of identical moving image by the user, because the feeling of moving 
image of each user who retrieves moving image and the way of understanding are different. Moreover, there is a disad- 
vantage that the time for choosing the keyword index of the desire is taken, even the system for inputting keyword index 
in which the user who retrieves moving image from keyword index information presented by a list type is made to 
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choose the keyword index of the desire. 

[0012] And, in the conventional method for retrieving moving image using the visual discrimination by the visual 
observation of the user, there is a disadvantage that hands are taken and the cost is high, because it is the method in 
which the user who retrieves moving image memorizes moving image that the user intends to retrieve and identify, and 

5 the user retrieves and identifies the moving image of the desire by observing all moving images. 

[0013] And, in the method for retrieving moving image using the number of continuous frames on the shot which 
constitutes moving image disclosed in Japanese Patent Publication JP7-46517, there is a disadvantage that it might be 
misunderstood that moving image is identical with moving image for retrieving, if the number of continuous frames of 
the moving image agrees even if the content of the moving image differs from the moving image for retrieving, because 

10 the number of continuous frames is not index information reflecting the content of the meaning of moving image. 

[0014] And, in the conventional real time discrimination method of image scene based on the time series coding of 
cutting, there is a disadvantage that the time for the extracting process of a cut point frame is taken and the cost also 
rises, because it is necessary to accurately extract a cut point frame in order to obtain a right collation result. 
[0015] And, in the apparatus for retrieving the cut point of moving image disclosed in Japanese Patent Publication 

15 No. JP7-236153, there is a disadvantage that it is impossible to retrieve and identify moving image of the desire auto- 
matically, because the detection of a cut point in moving image is a purpose, and it is only to detect a cut point by 
regarding the time variation of luminance difference and the time variation of color difference histogram correlation 
value. 

[0016] In the apparatus for processing moving image disclosed in Japanese Patent Publication No. JP7-1 52779, 
20 there is a disadvantage that time is taken due to the complicated process and the cost is high. Because the interim 
result which corresponds to the result of DCT operation by the process for reconstructing image data from compressed 
image data is made to be signals of the information of the feature quantity of moving image index, at least a kind of mov- 
ing image index is detected from this signals of the information of the feature quantity of moving image index, and the 
sum of difference value with the moving image index of examined image is processed at the threshold, and then moving 
25 image is detected as index image by the result of this threshold processing. 

SUMMARY OF THE INVENTION 

[0017] Accordingly, it is an object of the invention to provide a method and an apparatus for retrieving moving 
30 image, in which the information of the unique identification of moving image used for retrieving in accordance with the 
content of the moving image can be automatically extracted even in the computer with small resources such as PC, and 
in which moving image can be retrieved and identified automatically by using the information of the unique identification 
of moving image. 

[0018] According to the first feature of the invention, an apparatus for retrieving moving image to retrieve pre-deter- 
35 mined moving image from object moving image, comprises first extracting means for extracting the information of the 
feature quantity of each frame from the pre-determined moving image, second extracting means for extracting the infor- 
mation of the unique identification of moving image to identify the moving image in accordance with the information of 
the feature quantity, extracted by the first extracting means, and retrieving means for retrieving the pre-determined mov- 
ing image from the object moving image by comparing the information of the unique identification, extracted by the sec- 
40 ond extracting means, with the information of the unique identification prepared in accordance with the object moving 
image. 

[0019] According to the second feature of the invention, a method for retrieving moving image to retrieve pre-deter- 
mined moving image from object moving image, comprises the steps of (A) extracting the information of the feature 
quantity of each frame from the pre-determined moving image, (B) extracting the information of the unique identification 
45 of moving image to identify the moving image in accordance with the information of the feature quantity, extracted by 
said step (A), and (C) retrieving the pre-determined moving image from the object moving image by comparing the infor- 
mation of the unique identification, extracted by the step (B), with the information of the unique identification prepared 
in accordance with the object moving image. 

[0020] According to the third feature of the invention, a program for retrieving moving image to retrieve pre-deter- 
50 mined moving image from object moving image, which is executed by computer system, comprises the steps of (A) 
extracting the information of the feature quantity of each frame from the pre-determined moving image, (B) extracting 
the information of the unique identification of moving image to identify the moving image in accordance with the infor- 
mation of the feature quantity, extracted by the step (A), and (C) retrieving the pre-determined moving image from the 
object moving image by comparing the information of the unique identification, extracted by the step (B), with the infor- 
55 mation of the unique identification prepared in accordance with the object moving image. 
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BRIEF DESCRIPTION OF THE DRAWINGS 

[0021] The invention will be described in more detail in conjunction with the appended drawings, wherein: 

5 FIG. 1 is a block diagram showing the apparatus for retrieving moving image according to the present invention; 

FIG. 2A is a flowchart of the process in the apparatus for retrieving moving image according to the present inven- 
tion; 

FIG. 2B is a flowchart of the process in the apparatus for retrieving moving image according to the present inven- 
tion; 

10 FIG. 3 is a block diagram showing the apparatus for retrieving moving image according to the present invention; 

FIG. 4A is a flowchart of the process in the apparatus for retrieving moving image according to the present inven- 
tion; 

FIG. 4B is a flowchart of the process in the apparatus for retrieving moving image according to the present inven- 
tion; 

15 FIG. 5 is a block diagram showing the apparatus for retrieving moving image according to the present invention; 

FIG. 6A is a flowchart of the process in the apparatus for retrieving moving image according to the present inven- 
tion; 

FIG. 6B is a flowchart of the process in the apparatus for retrieving moving image according to the present inven- 
tion; 

20 FIG. 7 is a block diagram showing the apparatus for retrieving moving image according to the present invention; 

FIG. 8A is a flowchart of the process in the apparatus for retrieving moving image according to the present inven- 
tion; 

FIG. 8B is a flowchart of the process in the apparatus for retrieving moving image according to the present inven- 
tion; 

25 FIG. 9 is a block diagram showing the apparatus for retrieving moving image according to the present invention; 

FIG. 1 0A is a flowchart of the process in the apparatus for retrieving moving image according to the present inven- 
tion; 

FIG. 1 0B is a flowchart of the process in the apparatus for retrieving moving image according to the present inven- 
tion; 

30 FIG. 11 is a block diagram showing the apparatus for retrieving moving image according to the present invention; 

FIG. 12A is a flowchart of the process in the apparatus for retrieving moving image according to the present inven- 
tion; 

FIG. 12B is a flowchart of the process in the apparatus for retrieving moving image according to the present inven- 
tion; 

35 FIG. 13 is a block diagram showing the apparatus for retrieving moving image according to the present invention; 

FIG. 14 is a conceptual view showing the process of DCT (discrete cosine transform) operation in MPEG; 
FIG. 15A is a flowchart of the process in the apparatus for retrieving moving image according to the present inven- 
tion; 

FIG. 15B is a flowchart of the process in the apparatus for retrieving moving image according to the present inven- 
40 tion; 

FIG. 16 is a block diagram showing the apparatus for retrieving moving image according to the present invention; 
FIG. 17A is a flowchart of the process in the apparatus for retrieving moving image according to the present inven- 
tion; 

FIG. 17B is a flowchart of the process in the apparatus for retrieving moving image according to the present inven- 
45 tion; 

FIG. 18 is a block diagram showing the apparatus for retrieving moving image according to the present invention; 
FIG. 19A is a flowchart of the process in the apparatus for retrieving moving image according to the present inven- 
tion; 

FIG. 19B is a flowchart of the process in the apparatus for retrieving moving image according to the present inven- 
so tion; 

FIG. 20 is a block diagram showing the apparatus for retrieving moving image according to the present invention; 
FIG. 21 A is a flowchart of the process in the apparatus for retrieving moving image according to the present inven- 
tion; 

FIG. 21 B is a flowchart of the process in the apparatus for retrieving moving image according to the present inven- 
55 tion; 

FIG. 22 is a block diagram showing the apparatus for retrieving moving image according to the present invention; 
FIG. 23A is a flowchart of the process in the apparatus for retrieving moving image according to the present inven- 
tion; 



4 



EP 0 996 289 A2 



FIG. 23B is a flowchart of the process in the apparatus for retrieving moving image according to the present inven- 
tion; 

FIG. 24 is a block diagram showing the apparatus for retrieving moving image according to the present invention; 
FIG. 25A is a flowchart of the process in the apparatus for retrieving moving image according to the present inven- 
5 tion; 

FIG. 25B is a flowchart of the process in the apparatus for retrieving moving image according to the present inven- 
tion; 

FIG. 26 is a block diagram showing the apparatus for retrieving moving image according to the present invention; 
FIG. 27A is a flowchart of the process in the apparatus for retrieving moving image according to the present inven- 
10 tion; 

FIG. 27B is a flowchart of the process in the apparatus for retrieving moving image according to the present inven- 
tion; 

FIG. 28 is a block diagram showing the apparatus for retrieving moving image according to the present invention; 
FIG. 29A is a flowchart of the process in the apparatus for retrieving moving image according to the present inven- 
ts tion; 

FIG. 29B is a flowchart of the process in the apparatus for retrieving moving image according to the present inven- 
tion; 

FIG. 30 is a block diagram showing the apparatus for retrieving moving image according to the present invention; 
FIG. 31 A is a flowchart of the process in the apparatus for retrieving moving image according to the present inven- 
20 tion; 

FIG. 31 B is a flowchart of the process in the apparatus for retrieving moving image according to the present inven- 
tion; 

FIG. 32 is a block diagram showing the apparatus for retrieving moving image according to the present invention; 
FIG. 33A is a flowchart of the process in the apparatus for retrieving moving image according to the present inven- 
25 tion; 

FIG. 33B is a flowchart of the process in the apparatus for retrieving moving image according to the present inven- 
tion; 

FIG. 34 is a block diagram showing the apparatus for retrieving moving image according to the present invention; 
FIG. 35A is a flowchart of the process in the apparatus for retrieving moving image according to the present inven- 
30 tion; 

FIG. 35B is a flowchart of the process in the apparatus for retrieving moving image according to the present inven- 
tion; 

FIG. 36 is a block diagram showing the apparatus for retrieving moving image according to the present invention; 
FIG. 37A is a flowchart of the process in the apparatus for retrieving moving image according to the present inven- 
35 tion; 

FIG. 37B is a flowchart of the process in the apparatus for retrieving moving image according to the present inven- 
tion; 

FIG. 38 is a block diagram showing the apparatus for retrieving moving image according to the present invention; 
FIG. 39A is a flowchart of the process in the apparatus for retrieving moving image according to the present inven- 
40 tion; 

FIG. 39B is a flowchart of the process in the apparatus for retrieving moving image according to the present inven- 
tion; 

FIG. 40 is a block diagram showing the apparatus for retrieving moving image according to the present invention; 
FIG. 41 A is a flowchart of the process in the apparatus for retrieving moving image according to the present inven- 
45 tion; 

FIG. 41 B is a flowchart of the process in the apparatus for retrieving moving image according to the present inven- 
tion; 

FIG. 42 is a block diagram showing the apparatus for retrieving moving image according to the present invention; 
FIG. 43A is a flowchart of the process in the apparatus for retrieving moving image according to the present inven- 
50 tion; 

FIG. 43B is a flowchart of the process in the apparatus for retrieving moving image according to the present inven- 
tion; 

FIG. 44 is a block diagram showing the apparatus for retrieving moving image according to the present invention; 
and 

55 FIG. 45A is a flowchart of the process in the apparatus for retrieving moving image according to the present inven- 

tion. 

FIG. 45B is a flowchart of the process in the apparatus for retrieving moving image according to the present inven- 
tion; 
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FIG. 46 is a block diagram showing the apparatus for monitoring television commercials according to the present 
invention; and 

FIG. 47A is a flowchart of the process in the apparatus for monitoring television commercials according to the 
present invention. 

FIG. 47B is a flowchart of the process in the apparatus for retrieving moving image according to the present inven- 
tion; 

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS 

[0022] In the following, a method and an apparatus for retrieving moving image, and a recording medium according 
to the invention are explained in details referring to the drawings. 

[0023] An apparatus and a method for retrieving moving image in the first preferred embodiment according to the 
invention will be explained in FIGs. 1, 2A and 2B. 

[0024] FIG. 1 is a block diagram showing an apparatus for retrieving moving image according to the present inven- 
tion. In FIG. 1 , the apparatus for retrieving moving image comprises an extracting unit 102 for extracting information 103 
of the feature quantity of moving image, which includes physical information of the feature quantity of moving image 
such as information of luminance and color difference (Y, Cb, Cr) of each frame and / or information of color (R, G, B) 
of each frame, from moving image data 101, an extracting unit 104 for arranging at least one physical information 
included in the information 103 (such as the information of the luminance and the color difference (Y, Cb, Cr) of each 
frame and / or the information of the color (R, G, B) of each frame) on the time axis and for extracting information 105 
of the unique identification of moving image to identify the moving image from the arrangement of the physical informa- 
tion on the time axis, a storing unit 106 for storing the information 105 of the unique identification of the moving image 
extracted by the extracting unit 104, a retrieving unit 109 for retrieving identical information with the information 105 of 
the unique identification of the moving image stored in the storing unit 106 from information 108 of the unique identifi- 
cation of moving image beforehand extracted from other moving image data, a storing unit 1 1 0 for storing the retrieving 
result provided by the retrieving unit 109, and an output unit 111 for displaying and / or printing the retrieving result 
stored in the storing unit 1 10. On the apparatus for retrieving the moving image according to the present invention con- 
stituted like the above, an operation will be explained in the following. 

[0025] FIGs. 2A and 2B are flowcharts of the process in the apparatus for retrieving the moving image according to 
the present invention described in FIG. 1. In FIGs. 1, 2A and 2B, at first the extracting unit 102 for extracting the infor- 
mation of the feature quantity of the moving image fetches the moving image data 1 01 to be a retrieval key for identifying 
the moving image (in the step 201). Next, the extracting unit 102 extracts the information 103 of the feature quantity of 
the moving image, which includes the physical information of the feature quantity of the moving image such as the infor- 
mation of the luminance and the color difference (Y, Cb, Cr) of each frame and / or the information of the color (R, G, B) 
of each frame, from the moving image data 101 (in the step 202). 

[0026] Next, the extracting unit 104 for extracting the information of the unique identification of the moving image 
arranges at least one value from values of the physical information of the feature quantity of the moving image (such as 
the information of the luminance and the color difference (Y, Cb, Cr) of each frame and / or the information of the color 
(R, G, B) of each frame) based on the information 103, mean value of the values, total value of the values and / or dif- 
ference value of the values on the time axis. Then, the extracting unit 1 04 picks out all values arranged on the time axis, 
or some values from the values arranged on the time axis at uniform intervals or irregular intervals. And then, the 
extracting unit 1 04 extracts the picked out values as the information 1 05 of the unique identification of the moving image 
to be used for identifying the moving image (in the step 203). 

[0027] The storing unit 106 for storing the information of the unique identification of the moving image stores the 
information 105 of the unique identification of the moving image extracted by the extracting unit 104 (in the step 204). 
[0028] On the other hand, the information 108 of the unique identification of the moving image is extracted from 
other moving image data (the object moving image data for retrieving) (in the step 205). This extracting process can be 
executed by the processing equal to the foregoing steps 201 to 204. 

[0029] Then, the retrieving unit 109 for retrieving the information of the unique identification of the moving image 
compares the information 105 of the unique identification of the moving image stored in the storing unit 106 with the 
information 1 08 of the unique identification of the moving image extracted from the other moving image data (in the step 
206). 

[0030] This comparing result is stored in the storing unit 1 1 0 for storing the retrieving result (in the step 207). In the 
case of displaying and / or printing the comparing result (in the step 208), it is outputted by the output unit 111 for dis- 
playing and / or printing the retrieving result (in the step 209). 

[0031 ] The process of the above mentioned steps 206 to 209 is executed for all of the information 1 08 of the unique 
identification of the moving image extracted from the other moving image data (in the step 210). 

[0032] The apparatus and the method for retrieving the moving image according to the invention are explained the 
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above. The above mentioned extracting unit 104 arranges at least one value from values of the physical information of 
the feature quantity of the moving image (such as the information of the luminance and the color difference (Y, Cb, Cr) 
of each frame and the information of the color (R, G, B) of each frame) based on the information 103 of the feature 
quantity of the moving image, mean value of the values, total value of the values and / or difference value of the values 

5 on the time axis, and picks out all values arranged on the time axis, or some values from the values arranged on the 
time axis at uniform intervals or irregular intervals, and then extracts the picked out values as the information 1 05 of the 
unique identification of the moving image to be used for identifying the moving image. However, the above mentioned 
extracting unit 1 04 may arrange at least one value from values of a part or multiple parts of the physical information of 
the feature quantity of the moving image (such as the information of the luminance and the color difference (Y, Cb, Cr) 

10 of each frame and the information of the color (R, G, B) of each frame) based on the information 103 of the feature 
quantity of the moving image, mean value of the values, total value of the values and / or difference value of the values 
on the time axis, and pick out all values arranged on the time axis, or some values from the values arranged on the time 
axis at uniform intervals or irregular intervals, and then extract the picked out values as the information 105 of the 
unique identification of the moving image to be used for identifying the moving image. That is to say, values of informa- 

15 tion of feature quantity of moving image in each block unit may be calculated by dividing a frame into several blocks, 
and the information 1 05 of the unique identification of the moving image may be extracted by using all of the above men- 
tioned values of the information of the feature quantity of the moving image. And also, the information 1 05 of the unique 
identification of the moving image may be extracted by using some values of the information of the feature quantity of 
the moving image. As such embodiment, for example, values of the information of the feature quantity of the moving 

20 image, which are calculated from blocks that are located on the part except for the part of time display and the part of 
around frame (the part with unevenness), are used. 

[0033] And more, the extracting unit 104 may calculate values after executing the weighting on the physical infor- 
mation of the feature quantity of the moving image (such as the information of the luminance and the color difference 
(Y, Cb, Cr) of each frame and / or the information of the color (R, G, B) of each frame) based on the information 103, 
25 then may apply the values, mean value of the values, total value of the values and / or difference value of the values. 
[0034] Moreover, the extracting unit 104 may calculate values of the physical information of the feature quantity of 
the moving image, mean value of the values, total value of the values and / or difference value of the values, and then 
may quantize the calculated values. 

[0035] As mentioned above, according to the first embodiment of the invention, information of the unique identifica- 
30 tion of moving image for retrieving moving image depends on the content of the moving image can be extracted, con- 
sequently, identical moving image can be retrieved and identified from the moving image by using the information of the 
unique identification of the moving image at high speed. 

[0036] An apparatus and a method for retrieving moving image in the second preferred embodiment according to 
the invention will be explained in FIGs. 3, 4A and 4B. 
35 [0037] FIG. 3 is a block diagram showing an apparatus for retrieving moving image according to the present inven- 
tion. Still, an identical code is appended to the thing having the identical composition in FIG. 3 with FIG. 1. In FIG. 3, 
the apparatus for retrieving moving image comprises an extracting unit 302 for extracting information 303 of the feature 
quantity of compressed moving image, which includes DCT (Discrete Cosine Transform) coefficients of each frame and 
information of motion compensation of each frame, from compressed moving image data 301 provided by the standard 
40 compression encoding process for moving image such as MPEG, an extracting unit 304 for arranging at least one infor- 
mation of feature quantity from the DCT coefficients of each frame and the information of the motion compensation of 
each frame, which are included in the information 303, on the time axis and for extracting information 305 of the unique 
identification of compressed moving image to identify the compressed moving image from the arrangement of the infor- 
mation of the feature quantity on the time axis, a storing unit 306 for storing the information 305 of the unique identifi- 
es cation of the compressed moving image extracted by the extracting unit 304, a retrieving unit 309 for retrieving identical 
information with the information 305 of the unique identification of the compressed moving image stored in the storing 
unit 306 from information 308 of the unique identification of compressed moving image beforehand extracted from other 
moving image data, a storing unit 1 10 for storing the retrieving result provided by the retrieving unit 309, and an output 
unit 111 for displaying and / or printing the retrieving result stored in the storing unit 1 1 0. On the apparatus for retrieving 
so the moving image according to the present invention constituted like the above, an operation will be explained in the 
following. 

[0038] FIGs. 4A and 4B are flowcharts of the process in the apparatus for retrieving the moving image according to 
the present invention described in FIG. 3. Still, an identical step number is appended to the step having the identical 
process in FIGs. 4A and 4B with FIGs. 2A and 2B. In FIGs. 3, 4A and 4B, at first the extracting unit 302 for extracting 
55 the information of the feature quantity of the compressed moving image fetches the compressed moving image data 
301 to be a retrieval key for identifying the compressed moving image (in the step 401). Next, the extracting unit 302 
extracts the information 303 of the feature quantity of the compressed moving image, which includes the DCT coeffi- 
cients of each frame and the information of the motion compensation of each frame, from the compressed moving 
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image data 301 (in the step 402). 

[0039] Next, the extracting unit 304 for extracting the information of the unique identification of the compressed 
moving image calculates values of physical information of the feature quantity of the compressed moving image from 
the DCT coefficients of each frame, which is included in the information 303, mean value of the values, total value of 

5 the values and / or difference value of the values. On the other hand, the extracting unit 304 calculates values such as 
a motion vector, a mean motion vector between frames before and behind, a total motion vector, a difference vector and 
/ or a motion vector on a whole frame from the information of the motion compensation. And the extracting unit 304 
arranges at least one value from the above mentioned values on the time axis and then picks out all values arranged 
on the time axis, or some values from the values arranged on the time axis at uniform intervals or irregular intervals. 

10 And then, the extracting unit 304 extracts the picked out values as the information 305 of the unique identification of the 
compressed moving image to be used for identifying the compressed moving image (in the step 403). 
[0040] The storing unit 306 for storing the information of the unique identification of the compressed moving image 
stores the information 305 of the unique identification of the compressed moving image extracted by the extracting unit 
304 (in the step 404). 

15 [0041] On the other hand, the information 308 of the unique identification of the compressed moving image is 
extracted from other compressed moving image data (the object compressed moving image data for retrieving) (in the 
step 405). This extracting process can be executed by the processing equal to the foregoing steps 401 to 404. 
[0042] Then, the retrieving unit 309 for retrieving the information of the unique identification of the compressed 
moving image compares the information 305 of the unique identification of the compressed moving image stored in the 

20 storing unit 306 with the information 308 of the unique identification of the compressed moving image extracted from 
the other moving image data (in the step 406). 

[0043] This comparing result is stored in the storing unit 1 1 0 for storing the retrieving result (in the step 207). In the 
case of displaying and / or printing the comparing result (in the step 208), it is outputted by the output unit 111 for dis- 
playing and / or printing the retrieving result (in the step 209). 

25 [0044] The process of the above mentioned steps 406 to 209 is executed for all of the information 308 of the unique 
identification of the compressed moving image extracted from the other moving image data (in the step 410). 
[0045] The apparatus and the method for retrieving the moving image according to the invention are explained the 
above. The above mentioned extracting unit 304 calculates values of the physical information of the feature quantity of 
the compressed moving image from the DCT coefficients of each frame, which is included in the information 303, mean 

30 value of the values, total value of the values and / or difference value of the values, and values such as a motion vector, 
a mean motion vector between frames before and behind, a total motion vector, a difference vector and / or a motion 
vector on the whole frame from the information of the motion compensation. And the above mentioned extracting unit 
304 arranges at least one value from the above mentioned values and then picks out all values arranged on the time 
axis, or some values from the values arranged on the time axis at uniform intervals or irregular intervals. And then, the 

35 above mentioned extracting unit 304 extracts the picked out values as the information 305 of the unique identification 
of the compressed moving image to be used for identifying the compressed moving image. However, the above men- 
tioned extracting unit 304 may calculate values of a part or multiple parts of the physical information of the feature quan- 
tity of the compressed moving image from the DCT coefficients of each frame, which is included in the information 303, 
mean value of the values, total value of the values and / or difference value of the values, and values such as a motion 

40 vector, a mean motion vector between frames before and behind, a total motion vector, a difference vector and / or a 
motion vector on the whole frame from a part or multiple parts of the information of the motion compensation. That is 
to say, values of information of feature quantity of moving image in each block unit may be calculated by dividing a frame 
into several blocks, and the information 305 of the unique identification of the moving image may be extracted by using 
all of the above mentioned values of the information of the feature quantity of the moving image. And also, the informa- 

45 tion 305 of the unique identification of the moving image may be extracted by using some values of the information of 
the feature quantity of the moving image. As such embodiment, for example, values of the information of the feature 
quantity of the moving image, which are calculated from blocks that are located on the part except for the part of time 
display and the part of around frame (the part with unevenness), are used. 

[0046] And more, the extracting unit 304 may execute the weighting on the physical information of the feature quan- 
so tity of the compressed moving image from the DCT coefficients of each frame, which is included in the information 303, 
then may calculate values from the weighted information of the feature quantity of the compressed moving image, mean 
value of the values, total value of the values and / or difference value of the values. On the other hand, the extracting 
unit 304 may execute the weighting on the information of the motion compensation and may calculate a motion vector, 
a mean motion vector between frames before and behind, a total motion vector, a difference vector and / or a motion 
55 vector on a whole frame from the weighted information of the motion compensation. 

[0047] Moreover, in the extracting unit 304, values provided by quantizing the above mentioned values of the phys- 
ical information of the feature quantity of the compressed moving image and the like, and values provided by quantizing 
the above mentioned motion vectors, and the like may be applied. 
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[0048] As mentioned above, according to the second embodiment of the invention, information of the unique iden- 
tification of moving image for retrieving moving image depends on the content of the compressed moving image pro- 
vided by the standard compression encoding process for moving image such as MPEG can be extracted, consequently, 
identical moving image can be retrieved and identified from the moving image by using the information of the unique 
5 identification of the moving image at high speed. 

[0049] An apparatus and a method for retrieving moving image in the third preferred embodiment according to the 
invention will be explained in FIGs. 5, 6A and 6B. 

[0050] FIG. 5 is a block diagram showing an apparatus for retrieving moving image according to the present inven- 
tion. Still, an identical code is appended to the thing having the identical composition in FIG. 5 with FIG. 1. In FIG. 5, 

10 the apparatus for retrieving moving image comprises an extracting unit 1 02 for extracting information 1 03 of the feature 
quantity of moving image, which includes physical information of the feature quantity of moving image such as informa- 
tion of luminance and color difference (Y, Cb, Cr) of each frame and / or information of color (R, G, B) of each frame, 
from moving image data 1 01 , a detecting unit 501 for detecting a cut point of the moving image data 1 01 as information 
502 of a cut point, an extracting unit 503 for arranging at least one physical information included in the information 103 

15 (such as the information of the luminance and the color difference (Y, Cb, Cr) of each frame and / or the information of 
the color (R, G, B) of each frame) on the time axis and for extracting information 505 of the unique identification of mov- 
ing image to identify the moving image in each shot unit of the moving image data 1 01 divided by using the information 
502 of the cut point from the arrangement of the physical information on the time axis, a storing unit 1 06 for storing the 
information 505 of the unique identification of the moving image extracted by the extracting unit 503, a retrieving unit 

20 109 for retrieving identical information with the information 505 of the unique identification of the moving image stored 
in the storing unit 106 from information 508 of the unique identification of moving image beforehand extracted from 
other moving image data, a storing unit 110 for storing the retrieving result provided by the retrieving unit 109, and an 
output unit 111 for displaying and / or printing the retrieving result stored in the storing unit 110. On the apparatus for 
retrieving the moving image according to the present invention constituted like the above, an operation will be explained 

25 in the following. 

[0051] FIGs. 6A and 6B are flowcharts of the process in the apparatus for retrieving the moving image according to 
the present invention described in FIG. 5. Still, an identical step number is appended to the step having the identical 
process in FIGs. 6A and 6B with FIGs. 2A and 2B. In FIGs. 5, 6A and 6B, at first the extracting unit 102 for extracting 
the information of the feature quantity of the moving image and the detecting unit 501 for detecting the cut point fetch 

30 the moving image data 101 to be a retrieval key for identifying the moving image (in the step 601). 

[0052] Next, the extracting unit 1 02 extracts the information 1 03 of the feature quantity of the moving image, which 
includes the physical information of the feature quantity of the moving image such as the information of the luminance 
and the color difference (Y, Cb, Cr) of each frame and / or the information of the color (R, G, B) of each frame, from the 
moving image data 101 (in the step 202). 

35 [0053] And, the detecting unit 501 detects the cut point of the moving image data 1 01 as the information 502 of the 
cut point (in the step 602). This cut point of the moving image data 101 shows a point where the scene of the moving 
image data 1 01 changes or a point where the camera for photographing the moving image data 1 01 switches. 
[0054] Next, the extracting unit 503 for extracting the information of the unique identification of the moving image 
arranges at least one value from values of the physical information of the feature quantity of the moving image (such as 

40 the information of the luminance and the color difference (Y, Cb, Cr) of each frame and / or the information of the color 
(R, G, B) of each frame) based on the information 1 03, mean value of the values, total value of the values and / or dif- 
ference value of the values on the time axis. Then, the extracting unit 503 picks out all values arranged on the time axis, 
or some values from the values arranged on the time axis at uniform intervals or irregular intervals. And then, the 
extracting unit 503 extracts the picked out values in each shot unit of the moving image data 101 divided by the infor- 

45 mation 502 of the cut point as the information 505 of the unique identification of the moving image to be used for iden- 
tifying the moving image (in the step 603). 

[0055] The storing unit 106 for storing the information of the unique identification of the moving image stores the 
information 505 of the unique identification of the moving image extracted by the extracting unit 503 (in the step 604). 
[0056] On the other hand, the information 508 of the unique identification of the moving image is extracted from 
50 other moving image data (the object moving image data for retrieving) (in the step 205). This extracting process can be 
executed by the processing equal to the foregoing steps 601 to 604. 

[0057] Then, the retrieving unit 109 for retrieving the information of the unique identification of the moving image 
compares the information 505 of the unique identification of the moving image stored in the storing unit 106 with the 
information 508 of the unique identification of the moving image extracted from the other moving image data (in the step 
55 206). 

[0058] This comparing result is stored in the storing unit 1 1 0 for storing the retrieving result (in the step 207). In the 
case of displaying and / or printing the comparing result (in the step 208), it is outputted by the output unit 111 for dis- 
playing and / or printing the retrieving result (in the step 209). 
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[0059] The process of the above mentioned steps 206 to 209 is executed for all of the information 508 of the unique 
identification of the moving image extracted from the other moving image data (in the step 210). 

[0060] The apparatus and the method for retrieving the moving image according to the invention are explained the 
above. The above mentioned extracting unit 503 for extracting the information of the unique identification of the moving 

5 image arranges at least one value from values of the physical information of the feature quantity of the moving image 
(such as the information of the luminance and the color difference (Y, Cb, Cr) of each frame and / or the information of 
the color (R, G, B) of each frame) based on the information 1 03 of the feature quantity of the moving image, mean value 
of the values, total value of the values and / or difference value of the values on the time axis. However, the extracting 
unit 503 may arranges at least one value from values of a part or multiple parts of the physical information of the feature 

10 quantity of the moving image (such as the information of the luminance and the color difference (Y, Cb, Cr) of each 
frame and / or the information of the color (R, G, B) of each frame) based on the information 1 03 of the feature quantity 
of the moving image, mean value of the values, total value of the values and/ or difference value of the values on the 
time axis. That is to say, values of information of feature quantity of moving image in each block unit may be calculated 
by dividing a frame into several blocks, and the information 505 of the unique identification of the moving image may be 

15 extracted by using all of the above mentioned values of the information of the feature quantity of the moving image. And 
also, the information 505 of the unique identification of the moving image may be extracted by using some values of the 
information of the feature quantity of the moving image. As such embodiment, for example, values of the information of 
the feature quantity of the moving image, which are calculated from blocks that are located on the part except for the 
part of time display and the part of around frame (the part with unevenness), are used. 

20 [0061] And more, the extracting unit 503 may calculate values after executing the weighting on the physical infor- 
mation of the feature quantity of the moving image (such as the information of the luminance and the color difference 
(Y, Cb, Cr) of each frame and / or the information of the color (R, G, B) of each frame) based on the information 1 03 of 
the feature quantity of the moving image, then may apply the values, mean value of the values, total value of the values 
and / or difference value of the values. 

25 [0062] Moreover, the extracting unit 503 may calculate values of the physical information of the feature quantity of 
the moving image, mean value of the values, total value of the values and / or difference value of the values, and then 
may quantize the calculated values. 

[0063] As mentioned above, according to the third embodiment of the invention, the information 505 of the unique 
identification of the moving image for retrieving the moving image depends on the content of the moving image can be 
30 extracted by the less information quantity with utilizing the time concept of a shot according to the information 502 of 
the cut point. And, identical moving image can be retrieved and identified from the moving image by using the informa- 
tion 505 of the unique identification of the moving image at the higher speed. 

[0064] An apparatus and a method for retrieving moving image in the fourth preferred embodiment according to the 
invention will be explained in FIGs. 7, 8A and 8B. 

35 [0065] FIG. 7 is a block diagram showing an apparatus for retrieving moving image according to the present inven- 
tion. Still, an identical code is appended to the thing having the identical composition in FIG. 7 with FIG. 3. In FIG. 7, 
the apparatus for retrieving moving image comprises an extracting unit 302 for extracting information 303 of the feature 
quantity of compressed moving image, which includes DCT (Discrete Cosine Transform) coefficients of each frame and 
information of motion compensation of each frame, from compressed moving image data 301 provided by the standard 

40 compression encoding process for moving image such as MPEG, a detecting unit 701 for detecting a cut point of the 
compressed moving image data 301 as information 702 of a cut point, an extracting unit 703 for arranging at least one 
information of feature quantity from the DCT coefficients of each frame and the information of the motion compensation 
of each frame, which are included in the information 303, on the time axis and for extracting information 705 of the 
unique identification of compressed moving image to identify the compressed moving image from the arrangement of 

45 the information of the feature quantity on the time axis in each shot unit of the compressed moving image data 301 
divided by using the information 702 of the cut point, a storing unit 306 for storing the information 705 of the unique iden- 
tification of the compressed moving image extracted by the extracting unit 703, a retrieving unit 309 for retrieving iden- 
tical information with the information 705 of the unique identification of the compressed moving image stored in the 
storing unit 306 from information 708 of the unique identification of compressed moving image beforehand extracted 

so from other moving image data, a storing unit 1 1 0 for storing the retrieving result provided by the retrieving unit 309, and 
an output unit 111 for displaying and / or printing the retrieving result stored in the storing unit 110. On the apparatus 
for retrieving the moving image according to the present invention constituted like the above, an operation will be 
explained in the following. 

[0066] FIGs. 8A and 8B are flowcharts of the process in the apparatus for retrieving the moving image according to 
55 the present invention described in FIG. 7. Still, an identical step number is appended to the step having the identical 
process in FIGs. 8A and 8B with FIGs. 4A and 4B. In FIGs. 7, 8A and 8B, at first the extracting unit 302 for extracting 
the information of the feature quantity of the compressed moving image and the detecting unit 701 for detecting the cut 
point fetch the compressed moving image data 301 to be a retrieval key for identifying the compressed moving image 
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(in the step 801). Next, the extracting unit 302 extracts the information 303 of the feature quantity of the compressed 
moving image, which includes the DCT coefficients of each frame and the information of the motion compensation of 
each frame, from the compressed moving image data 301 (in the step 402). And, the detecting unit 701 detects the cut 
point of the compressed moving image data 301 as the information 702 of the cut point (in the step 802). 

5 [0067] Next, the extracting unit 703 for extracting the information of the unique identification of the compressed 
moving image calculates values of physical information of the feature quantity of the compressed moving image from 
the DCT coefficients of each frame, which is included in the information 303, mean value of the values, total value of 
the values and / or difference value of the values. On the other hand, the extracting unit 703 calculates values such as 
a motion vector, a mean motion vector between frames before and behind, a total motion vector, a difference vector and 

10 / or a motion vector on a whole frame from the information of the motion compensation. And the extracting unit 703 
arranges at least one value from the above mentioned values on the time axis and then picks out all values arranged 
on the time axis, or some values from the values arranged on the time axis at uniform intervals or irregular intervals. 
And then, the extracting unit 703 extracts the picked out values in each shot unit of the compressed moving image data 
301 divided by using the information 702 of the cut point as the information 705 of the unique identification of the com- 

15 pressed moving image to be used for identifying the compressed moving image (in the step 803). 

[0068] The storing unit 306 for storing the information of the unique identification of the compressed moving image 
stores the information 705 of the unique identification of the compressed moving image extracted by the extracting unit 
703 (in the step 804). 

[0069] On the other hand, the information 708 of the unique identification of the compressed moving image is 
20 extracted from other compressed moving image data (the object compressed moving image data for retrieving) (in the 
step 405). This extracting process can be executed by the processing equal to the foregoing steps 801 to 804. 
[0070] Then, the retrieving unit 309 for retrieving the information of the unique identification of the compressed 
moving image compares the information 705 of the unique identification of the compressed moving image stored in the 
storing unit 306 with the information 708 of the unique identification of the compressed moving image extracted from 
25 the other moving image data (in the step 406). 

[0071] This comparing result is stored in the storing unit 1 1 0 for storing the retrieving result (in the step 207). In the 
case of displaying and / or printing the comparing result (in the step 208), it is outputted by the output unit 111 for dis- 
playing and / or printing the retrieving result (in the step 209). 

[0072] The process of the above mentioned steps 406 to 209 is executed for all of the information 708 of the unique 

30 identification of the compressed moving image extracted from the other moving image data (in the step 410). 

[0073] The apparatus and the method for retrieving the moving image according to the invention are explained the 
above. The above mentioned extracting unit 703 calculates values of the physical information of the feature quantity of 
the compressed moving image from the DCT coefficients of each frame, which is included in the information 303, mean 
value of the values, total value of the values and / or difference value of the values, and values such as a motion vector, 

35 a mean motion vector between frames before and behind, a total motion vector, a difference vector and / or a motion 
vector on a whole frame from the information of the motion compensation. However, the extracting unit 703 may calcu- 
late values of a part or multiple parts of the physical information of the feature quantity of the compressed moving image 
from the DCT coefficients of each frame, which is included in the information 303, mean value of the values, total value 
of the values and / or difference value of the values, and values such as motion vector, a mean motion vector between 

40 frames before and behind, a total motion vector, a difference vector and / or a motion vector on the whole frame from a 
part or multiple parts of the information of the motion compensation. That is to say, values of information of feature 
quantity of moving image in each block unit may be calculated by dividing a frame into several blocks, and the informa- 
tion 705 of the unique identification of the moving image may be extracted by using all of the above mentioned values 
of the information of the feature quantity of the moving image. And also, the information 705 of the unique identification 

45 of the moving image may be extracted by using some values of the information of the feature quantity of the moving 
image. As such embodiment, for example, values of the information of the feature quantity of the moving image, which 
are calculated from blocks that are located on the part except for the part of time display and the part of around frame 
(the part with unevenness), are used. 

[0074] And more, the extracting unit 703 may execute the weighting on the physical information of the feature quan- 
50 tity of the compressed moving image from the DCT coefficients of each frame, which is included in the information 303, 
then may calculate the above mentioned values from the weighted information of the feature quantity of the compressed 
moving image. On the other hand, the extracting unit 703 may execute the weighting on the information of the motion 
compensation and may calculate the above mentioned motion vectors from the weighted information of the motion com- 
pensation. 

55 [0075] Moreover, in the extracting unit 703, values provided by quantizing the above mentioned values of the phys- 
ical information of the feature quantity of the compressed moving image and values provided by quantizing the above 
mentioned motion vectors, and the like may be applied. 

[0076] As mentioned above, according to the fourth embodiment of the invention, the information 705 of the unique 
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identification of the compressed moving image for retrieving moving image depends on the content of the compressed 
moving image provided by the standard compression encoding process for moving image such as MPEG can be 
extracted by the less information quantity with utilizing the time concept of the shot according to the information 702 of 
the cut point. And, identical moving image can be retrieved and identified from the moving image by using the informa- 
5 tion 705 of the unique identification of the compressed moving image at the higher speed. 

[0077] An apparatus and a method for retrieving moving image in the fifth preferred embodiment according to the 
invention will be explained in FIGs. 9, 10A and 10B. 

[0078] FIG. 9 is a block diagram showing an apparatus for retrieving moving image according to the present inven- 
tion. Still, an identical code is appended to the thing having the identical composition in FIG. 9 with FIG. 1 and FIG. 3. 

10 In FIG. 9, the apparatus for retrieving moving image comprises an extracting unit 901 for extracting information of 
motion compensation in the MxN(M and N are natural numbers) pixels unit of moving image data 101, a DCT (Dis- 
crete Cosine Transform) operation unit 902 for obtaining a DCT coefficient by executing a DCT operation in the M' x N' 
(M' and N' are natural numbers) pixels unit of the moving image data 101, an extracting unit 302 for extracting informa- 
tion 303 of the feature quantity of compressed moving image, which includes DCT coefficients of each frame and infor- 

15 mation of motion compensation of each frame, from compressed moving image data 301 provided by the standard 
compression encoding process for moving image such as MPEG, an extracting unit 304 for arranging at least one infor- 
mation of feature quantity from the DCT coefficients of each frame and the information of the motion compensation of 
each frame, which are the information 303 extracted by the extracting unit 302, or by the extracting unit 901 and the 
DCT operation unit 902, on the time axis and for extracting information 305 of the unique identification of compressed 

20 moving image to identify the compressed moving image from the arrangement of the information of feature quantity on 
the time axis, a storing unit 306 for storing the information 305 of the unique identification of the compressed moving 
image extracted by the extracting unit 304, a retrieving unit 309 for retrieving identical information with the information 
305 of the unique identification of the compressed moving image stored in the storing unit 306 from information 308 of 
the unique identification of compressed moving image beforehand extracted from other moving image data, a storing 

25 unit 1 1 0 for storing the retrieving result provided by the retrieving unit 309, and an output unit 111 for displaying and / 
or printing the retrieving result stored in the storing unit 1 1 0. On the apparatus for retrieving the moving image according 
to the present invention constituted like the above, an operation will be explained in the following. 
[0079] FIGs. 1 0A and 1 0B are flowcharts of the process in the apparatus for retrieving the moving image according 
to the present invention described in FIG. 9. Still, an identical step number is appended to the step having the identical 

30 process in FIGs. 1 0A and 1 0B with FIGs. 4A and 4B. In FIGs. 9, 1 0A and 1 0B, at first whether inputted moving image 
data is compressed is judged, (in the step 1001). 

[0080] In case the inputted moving image data is the uncompressed moving image data 1 01 , the extracting unit 901 
for extracting the information of the motion compensation and the DCT operation unit 902 fetch the moving image data 
101 to be a retrieval key for identifying the moving image (in the step 1002). 

35 [0081] The extracting unit 901 divides each frame of the moving image data 1 01 into the M x N (M and N are natural 
numbers) pixels units, then calculates the information of the motion compensation in the M x N pixels unit (in the step 
1003). On the other hand, the DCT operation unit 902 divides each frame of the moving image data 101 into the M' x 
N' (M' and N' are natural numbers) pixels units, then calculates the DCT coefficient by executing the DCT operation in 
the M' x N' pixels unit (in the step 1004). 

40 [0082] The information of the motion compensation in the M x N pixels unit provided by the extracting unit 901 and 
the DCT coefficient in the M' x N' pixels unit provided by the DCT operation unit 902 are sent to the extracting unit 304 
for extracting the information of the unique identification of the compressed moving image as the information 303 of the 
feature quantity of the compressed moving image. 

[0083] On the other hand, in case the inputted moving image data is the compressed moving image data 301, the 
45 extracting unit 302 for extracting the information of the feature quantity of the compressed moving image fetches the 
compressed moving image data 301 to be a retrieval key for identifying the compressed moving image (in the step 401 ). 
And the extracting unit 302 extracts the information 303 of the feature quantity of the compressed moving image, which 
includes the DCT coefficients of each frame and the information of the motion compensation of each frame, from the 
compressed moving image data 301 (in the step 402). 
so [0084] Next, the extracting unit 304 for extracting the information of the unique identification of the compressed 
moving image calculates values of physical information of the feature quantity of the compressed moving image from 
the DCT coefficients of each frame, which is included in the information 303 extracted by extracting unit 302, or the 
extracting unit 901 and the DCT operation unit 902, mean value of the values, total value of the values and / or differ- 
ence value of the values. On the other hand, the extracting unit 304 calculates values such as a motion vector, a mean 
55 motion vector between frames before and behind, a total motion vector, a difference vector and / or a motion vector on 
a whole frame from the information of the motion compensation. And the extracting unit 304 arranges at least one value 
from the above mentioned values on the time axis and then picks out all values arranged on the time axis, or some val- 
ues from the values arranged on the time axis at uniform intervals or irregular intervals. And then, the extracting unit 
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304 extracts the picked out values as the information 305 of the unique identification of the compressed moving image 
to be used for identifying the compressed moving image (in the step 403). 

[0085] The storing unit 306 for storing the information of the unique identification of the compressed moving image 
stores the information 305 of the unique identification of the compressed moving image extracted by the extracting unit 
5 304 (in the step 404). 

[0086] On the other hand, the information 308 of the unique identification of the compressed moving image is 
extracted from other compressed moving image data (the object compressed moving image data for retrieving) (in the 
step 405). This extracting process can be executed by the processing equal to the foregoing steps 1001 to 404. 
[0087] Then, the retrieving unit 309 for retrieving the information of the unique identification of the compressed 
10 moving image compares the information 305 of the unique identification of the compressed moving image stored in the 
storing unit 306 with the information 308 of the unique identification of the compressed moving image extracted from 
the other moving image data (in the step 406). 

[0088] This comparing result is stored in the storing unit 1 1 0 for storing the retrieving result (in the step 207). In the 
case of displaying and / or printing the comparing result (in the step 208), it is outputted by the output unit 111 for dis- 

15 playing and / or printing the retrieving result (in the step 209). 

[0089] The process of the above mentioned steps 406 to 209 is executed for all of the information 308 of the unique 
identification of the compressed moving image extracted from the other moving image data (in the step 410). 
[0090] The apparatus and the method for retrieving the moving image according to the invention are explained the 
above. The M x N pixels unit and the M' x N' pixels unit may be made to be a value equal to the pixel unit of the standard 

20 compression encoding system such as MPEG. 

[0091] As mentioned above, according to the fifth embodiment of the invention, by executing the preprocessing (the 
steps 1003 and 1004) on the uncompressed moving image data 101, the information 305 of the unique identification of 
the compressed moving image for retrieving moving image depends on the content of the moving image 101 and the 
compressed moving image 301 can be extracted by the equal processing (the step 403). And identical moving image 

25 can be retrieved and identified from the moving image by using the information 305 of the unique identification of the 
compressed moving image at the higher speed. 

[0092] An apparatus and a method for retrieving moving image in the sixth preferred embodiment according to the 
invention will be explained in FIGs. 11, 1 2A and 1 2B. 

[0093] FIG. 1 1 is a block diagram showing an apparatus for retrieving moving image according to the present inven- 

30 tion. Still, an identical code is appended to the thing having the identical composition in FIG. 1 1 with FIG. 5, FIG. 7 and 
FIG. 9. In FIG. 1 1 , the apparatus for retrieving moving image comprises an extracting unit 901 for extracting information 
of motion compensation in the M x N (M and N are natural numbers) pixels unit of moving image data 1 01 , a DCT (Dis- 
crete Cosine Transform) operation unit 902 for obtaining a DCT coefficient by executing a DCT operation in the M' x N' 
(M' and NT are natural numbers) pixels unit of the moving image data 1 01 , a detecting unit 501 for detecting a cut point 

35 of the moving image data 1 01 as information 502 of a cut point, an extracting unit 302 for extracting information 303 of 
the feature quantity of compressed moving image, which includes DCT coefficients of each frame and information of 
motion compensation of each frame, from compressed moving image data 301 provided by the standard compression 
encoding process for moving image such as MPEG, a detecting unit 701 for detecting a cut point of the compressed 
moving image data 301 as information 702 of a cut point, an extracting unit 703 for arranging at least one information 

40 of feature quantity from the DCT coefficients of each frame and the information of the motion compensation of each 
frame, which are included in the information 303 extracted by the extracting unit 901 and the DCT operation unit 902, 
or the extracting unit 302, on the time axis and for extracting information 705 of the unique identification of compressed 
moving image to identify the compressed moving image from the arrangement of the information of the feature quantity 
on the time axis in each shot unit of the compressed moving image data 301 divided by using the information 702 of the 

45 cut point, a storing unit 306 for storing the information 705 of the unique identification of the compressed moving image 
extracted by the extracting unit 703, a retrieving unit 309 for retrieving identical information with the information 705 of 
the unique identification of the compressed moving image stored in the storing unit 306 from information 708 of the 
unique identification of compressed moving image beforehand extracted from other moving image data, a storing unit 
1 1 0 for storing the retrieving result provided by the retrieving unit 309, and an output unit 111 for displaying and / or 

50 printing the retrieving result stored in the storing unit 110. On the apparatus for retrieving the moving image according 
to the present invention constituted like the above, an operation will be explained in the following. 
[0094] FIGs. 12A and 12B are flowcharts of the process in the apparatus for retrieving the moving image according 
to the present invention described in FIG. 1 1 . Still, an identical step number is appended to the step having the identical 
process in FIGs. 12A and 12B with FIGs. 6A, 6B, 8A, 8B, 10A and 10B. In FIGs. 11, 12A and 12B, at first whether input- 

55 ted moving image data is compressed is judged (in the step 1 001 ). 

[0095] In case the inputted moving image data is the uncompressed moving image data 1 01 , the extracting unit 901 
for extracting the information of the motion compensation, the DCT operation unit 902 and the detecting unit 501 for 
detecting the cut point fetch the moving image data 1 01 to be a retrieval key for identifying the moving image (in the step 
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1201). 

[0096] The extracting unit 901 divides each frame of the moving image data 1 01 into the M x N (M and N are natural 
numbers) pixels units, then calculates the information of the motion compensation in the MxN pixels unit (in the step 
1003). On the other hand, the DCT operation unit 902 divides each frame of the moving image data 101 into the M' x 
5 N' (M' and N' are natural numbers) pixels units, then calculates the DCT coefficient by executing the DCT operation in 
the M' x N' pixels unit (in the step 1004). 

[0097] And, the detecting unit 501 detects the cut point of the moving image data 1 01 as the information 502 of the 
cut point (in the step 602). 

[0098] The information of the motion compensation in the MxN pixels unit provided by the extracting unit 901 and 
10 the DCT coefficient in the M' x N' pixels unit provided by the DCT operation unit 902 are sent to the extracting unit 703 
for extracting the information of the unique identification of the compressed moving image with the information 502 of 
the cut point detected by the detecting unit 501 as the information 303 of the feature quantity of the compressed moving 
image. 

[0099] On the other hand, in case the inputted moving image data is the compressed moving image data 301, the 
15 extracting unit 302 for extracting the information of the feature quantity of the compressed moving image and the detect- 
ing unit 701 for detecting the cut point fetch the compressed moving image data 301 to be a retrieval key for identifying 
the compressed moving image (in the step 801 ). Next, the extracting unit 302 extracts the information 303 of the feature 
quantity of the compressed moving image, which includes the DCT coefficients of each frame and the information of 
the motion compensation of each frame, from the compressed moving image data 301 (in the step 402). And, the 
20 detecting unit 701 detects the cut point of the compressed moving image data 301 as the information 702 of the cut 
point (in the step 802). 

[0100] Next, the extracting unit 703 for extracting the information of the unique identification of the compressed 
moving image calculates values of physical information of the feature quantity of the compressed moving image from 
the DCT coefficients of each frame, which is included in the information 303, mean value of the values, total value of 

25 the values and / or difference value of the values. On the other hand, the extracting unit 703 calculates values such as 
a motion vector, a mean motion vector between frames before and behind, a total motion vector, a difference vector and 
/ or a motion vector on a whole frame from the information of the motion compensation. And the extracting unit 703 
arranges at least one value from the above mentioned values on the time axis and then picks out all values arranged 
on the time axis, or some values from the values arranged on the time axis at uniform intervals or irregular intervals. 

30 And then, the extracting unit 703 extracts the picked out values in each shot unit of the moving image data 1 01 divided 
by using the information 502 or in each shot unit of the compressed moving image data 301 divided by using the infor- 
mation 702 as the information 705 of the unique identification of the compressed moving image to be used for identify- 
ing the compressed moving image (in the step 803). 

[0101] The storing unit 306 for storing the information of the unique identification of the compressed moving image 
35 stores the information 705 of the unique identification of the compressed moving image extracted by the extracting unit 
703 (in the step 804). 

[0102] On the other hand, the information 708 of the unique identification of the compressed moving image is 
extracted from other compressed moving image data (the object compressed moving image data for retrieving) (in the 
step 405). This extracting process can be executed by the processing equal to the foregoing steps 1001 to 804. 
40 [0103] Then, the retrieving unit 309 for retrieving the information of the unique identification of the compressed 
moving image compares the information 705 of the unique identification of the compressed moving image stored in the 
storing unit 306 with the information 708 of the unique identification of the compressed moving image extracted from 
the other moving image data (in the step 406). 

[0104] This comparing result is stored in the storing unit 1 1 0 for storing the retrieving result (in the step 207). In the 
45 case of displaying and / or printing the comparing result (in the step 208), it is outputted by the output unit 111 for dis- 
playing and / or printing the retrieving result (in the step 209). 

[0105] The process of the above mentioned steps 406 to 209 is executed for all of the information 708 of the unique 
identification of the compressed moving image extracted from the other moving image data (in the step 410). 
[0106] The apparatus and the method for retrieving the moving image according to the invention are explained the 
so above. The MxN pixels unit and the M' x N' pixels unit may be made to be a value equal to the pixel unit of the standard 
compression encoding system such as MPEG. 

[0107] As mentioned above, according to the sixth embodiment of the invention, by executing the preprocessing 
(the steps 1003 and 1004) on the uncompressed moving image data 101, the information 705 of the unique identifica- 
tion of the compressed moving image for retrieving moving image depends on the content of the moving image 1 01 and 
55 the compressed moving image 301 can be extracted by the equal processing (the step 803). And, the information 705 
of the unique identification of the compressed moving image for retrieving moving image can be extracted by the less 
information quantity with utilizing the time concept of the shot according to the information 502 and the information 702. 
Identical moving image can be retrieved and identified from the moving image by using the information 705 of the 
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unique identification of the compressed moving image at the higher speed. 

[0108] An apparatus and a method for retrieving moving image in the seventh preferred embodiment according to 
the invention will be explained in FIGs. 13 to 15B. 

[0109] FIG. 13 is a block diagram showing an apparatus for retrieving moving image according to the present inven- 

5 tion. Still, an identical code is appended to the thing having the identical composition in FIG. 13 with FIG. 3 and FIG. 9. 
In FIG. 13, the apparatus for retrieving moving image comprises an extracting unit 901 for extracting information of 
motion compensation in the M x N (M and N are natural numbers) pixels unit of moving image data 101 , an extracting 
unit 1 301 for calculating an average pixel value, such as information of average color (R, G, B) or information of average 
luminance and average color difference (Y, Cb, Cr), in the M' x N' (M' and N' are natural numbers) pixels unit, an extract- 

10 ing unit 1302 for extracting information 1303 of the feature quantity of compressed moving image, which includes DC 
(Direct Current) components of DCT (Discrete Cosine Transform) coefficients of each frame and information of motion 
compensation of each frame, from compressed moving image data 301 provided by the standard compression encod- 
ing process for moving image such as MPEG, an extracting unit 1304 for arranging at least one information of feature 
quantity from the DC components of the DCT coefficients of each frame and the information of the motion compensa- 

15 tion of each frame or the average pixel value, which are included in the information 1303 extracted by the extracting unit 
901 and the extracting unit 1301 , or the extracting unit 1302, on the time axis and for extracting information 305 of the 
unique identification of compressed moving image to identify the compressed moving image from the arrangement of 
the information of the feature quantity on the time axis, a storing unit 306 for storing the information 305 of the unique 
identification of the compressed moving image extracted by the extracting unit 1304, a retrieving unit 309 for retrieving 

20 identical information with the information 305 of the unique identification of the compressed moving image stored in the 
storing unit 306 from information 308 of the unique identification of compressed moving image beforehand extracted 
from other moving image data, a storing unit 1 1 0 for storing the retrieving result provided by the retrieving unit 309, and 
an output unit 111 for displaying and / or printing the retrieving result stored in the storing unit 110. 
[0110] FIG. 14 is a conceptual scheme showing the relationship between the DCT (Discrete Cosine Transform) 

25 operation and the IDCT (Inverse Discrete Cosine Transform) operation, which are used in the international standard 
encoding system such as MPEG, and frame image. 

[0111] The expression of the DCT operation is shown in the following EXPRESSION 1. 

N-1 N-1 

30 F(u, v) = {2C (u) C (v) / N} £ £ cos { (2x+1 )un / 2N } cos { (2y+1 )vn / 2N } [EXPRESSION 1 ] 

x=0y=0 

[0112] And, the expression of the IDCT operation is shown in the following EXPRESSION 2. 

35 

N-1 N-1 

f(x, y) = (2 / N) £ £ C (u) C (v) cos { (2x+1 ) un / 2N } cos { (2y+1 ) V7t/2N } [EXPRESSION 2] 

u=0v=0 

40 

[0113] However, in the above mentioned EXPRESSIONS 1 and 2, 

"N" is showing a block size of intra-code; 
"x" and "y" are showing a coordinate in a block; 
45 "f (x ,y)" is showing a pixel value of the coordinate (x, y) in a block; 

"u" and "v" are showing a coordinate after the DCT conversion; 

"F (u, v)" is showing a DCT coefficient value of the coordinate (u, v) after the DCT conversion; 
"C (u)" is made 1/V 2, if u = 0; 
1 , if u * 0; and 
50 "C (v)" is made 1/V 2, if v = 0; 

1, if v*0. 

[0114] In FIG. 14, for example, in case the block size N of one frame is 8 and the coordinate (u, v) after the DCT 
conversion is (0, 0), F (u, v) in the EXPRESSION 1 is shown in the following EXPRESSION 3. 

55 

7 7 

F (0, 0) = ( 1 / 8) £ £ f (x, y) [EXPRESSION 3] 

x=0y=0 
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[0115] And more, an average pixel value in the block is shown in the following EXPRESSION 4. 

7 7 

Average Pixel Value in the block = £ £ f (x, y) / 8 x 8 [EXPRESSION 4] 

x=0y=0 

= F (0, 0) / 8 



[0116] As shown in the above mentioned EXPRESSION 3 and 4, the DC component F (u, v) of the DCT coefficient 
becomes a value provided by multiplying the average pixel value in the block by the quantization number (N) of the 
block. That is to say, the DC components F (u, v) of the DCT coefficients extracted by the extracting unit 1 302 for extract- 
ing the information of the feature quantity of the compressed moving image and the average pixel value provided by the 
extracting unit 1301 for extracting the average value of the block unit may be handled as identical information data con- 
sidering the quantization number of the block. Specifically, the information 1303 extracted by the extracting unit 1302, 
and the information 1303 depends on the information of the motion compensation extracted by the extracting unit 901 
and the average pixel value extracted by the extracting unit 1301 may be considered information of data having an iden- 
tical content composition. 

[0117] On the apparatus for retrieving the moving image according to the present invention constituted like the 
above, an operation will be explained in the following. 

[0118] FIGs. 15A and 15B are flowcharts of the process in the apparatus for retrieving the moving image according 
to the present invention described in FIG. 13. Still, an identical step number is appended to the step having the identical 
process in FIGs. 1 5A and 1 5B with FIGs. 4A, 4B, 1 0A and 1 0B. In FIGs. 1 3, 1 5A and 1 5B, at first whether inputted mov- 
ing image data is compressed is judged, (in the step 1001). 

[0119] In case the inputted moving image data is the uncompressed moving image data 1 01 , the extracting unit 901 
for extracting the information of the motion compensation and the extracting unit 1301 for extracting the average value 
of the block unit fetch the moving image data 101 to be a retrieval key for identifying the moving image (in the step 

1501) . 

[0120] The extracting unit 901 divides each frame of the moving image data 1 01 into the M x N (M and N are natural 
numbers) pixels units, then calculates the information of the motion compensation in the M x N pixels unit (in the step 
1 003). On the other hand, the extracting unit 1 301 divides each frame of the moving image data 101 into the M' x N' (IW 
and N' are natural numbers) pixels units, then calculates the average pixel value in the M' x N' pixels unit (in the step 

1502) . 

[0121] The information of the motion compensation in the MxN pixels unit provided by the extracting unit 901 and 
the average pixel value in the M' x N' pixels unit provided by the extracting unit 1301 are sent to the extracting unit 1304 
for extracting the information of the unique identification of the compressed moving image as the information 1303 of 
the feature quantity of the compressed moving image. 

[0122] On the other hand, in case the inputted moving image data is the compressed moving image data 301, the 
extracting unit 1302 for extracting the information of the feature quantity of the compressed moving image fetches the 
compressed moving image data 301 to be a retrieval key for identifying the compressed moving image (in the step 401 ). 
And the extracting unit 1302 extracts the information 1303 of the feature quantity of the compressed moving image, 
which includes the DC components of the DCT coefficients of each frame and the information of the motion compen- 
sation of each frame, from the compressed moving image data 301 (in the step 1503). 

[0123] Next, the extracting unit 1304 for extracting the information of the unique identification of the compressed 
moving image calculates values of physical information of the feature quantity of compressed moving image from the 
DC components of the DCT coefficients of each frame and / or values of the average pixel value of each frame, which 
are included in the information 1 303, mean value of the values, total value of the values and / or difference value of the 
values. On the other hand, the extracting unit 1304 calculates values such as a motion vector, a mean motion vector 
between frames before and behind, a total motion vector, a difference vector and / or a motion vector on a whole frame 
from the information of the motion compensation. And the extracting unit 1304 arranges at least one value from the 
above mentioned values on the time axis and then picks out all values arranged on the time axis, or some values from 
the values arranged on the time axis at uniform intervals or irregular intervals. And then, the extracting unit 1304 
extracts the picked out values as the information 305 of the unique identification of the compressed moving image to be 
used for identifying the compressed moving image (in the step 1504). 

[0124] The storing unit 306 for storing the information of the unique identification of the compressed moving image 
stores the information 305 of the unique identification of the compressed moving image extracted by the extracting unit 
1304 (in the step 404). 

[0125] On the other hand, the information 308 of the unique identification of the compressed moving image is 
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extracted from other compressed moving image data (the object compressed moving image data for retrieving) (in the 
step 405). This extracting process can be executed by the processing equal to the foregoing steps 1001 to 404. 
[0126] Then, the retrieving unit 309 for retrieving the information of the unique identification of the compressed 
moving image compares the information 305 of the unique identification of the compressed moving image stored in the 
5 storing unit 306 with the information 308 of the unique identification of the compressed moving image extracted from 
the other moving image data (in the step 406). 

[0127] This comparing result is stored in the storing unit 1 1 0 for storing the retrieving result (in the step 207). In the 
case of displaying and / or printing the comparing result (in the step 208), it is outputted by the output unit 111 for dis- 
playing and / or printing the retrieving result (in the step 209). 

10 [0128] The process of the above mentioned steps 406 to 209 is executed for all of the information 308 of the unique 
identification of the compressed moving image extracted from the other moving image data (in the step 410). 
[0129] The apparatus and the method for retrieving the moving image according to the invention are explained the 
above. The above mentioned extracting unit 1304 calculates values of physical information of the feature quantity of the 
compressed moving image from the DC components of the DCT coefficients of each frame and / or values of the aver- 

15 age pixel value of each frame, which are included in the information 1303, mean value of the values, total value of the 
values and / or difference value of the values. And the extracting unit 1304 calculates values such as a motion vector, 
a mean motion vector between frames before and behind, a total motion vector, a difference vector and / or a motion 
vector on a whole frame from the information of the motion compensation. However, the extracting unit 1304 may cal- 
culate the above mentioned values by using a part or multiple parts of the physical information of the feature quantity 

20 of the compressed moving image from the DC components of the DCT coefficients of each frame and / or the average 
pixel value of each frame, which are included in the information 1303, and the above mentioned vectors by using a part 
or multiple parts of the information of the motion compensation. 

[0130] And more, the extracting unit 1304 may execute the weighting on the physical information of the feature 
quantity of the compressed moving image from the DC components of the DCT coefficients of each frame and / or on 
25 the average pixel value of each frame, then may calculate the above mentioned values from the weighted information 
of the feature quantity of the compressed moving image and / or the weighted average pixel value. On the other hand, 
the extracting unit 1304 may execute the weighting on the information of the motion compensation and may calculate 
the above mentioned vectors from the weighted information of the motion compensation. 

[0131] Moreover, in the extracting unit 1304, values provided by quantizing the above mentioned values of the phys- 
30 ical information or of the average pixel value or the like, and values provided by quantizing the above mentioned motion 
vectors and the like may be applied. 

[0132] Further more, the M x N pixels unit and the M' x N' pixels unit may be made to be a value equal to the pixel 
unit of the standard compression encoding system such as MPEG. 

[0133] As mentioned above, according to the seventh embodiment of the invention, by executing the preprocessing 
35 on the moving image, the information 305 of the unique identification of the compressed moving image for retrieving the 
moving image depends on the content of the moving image data 1 01 and the compressed moving image data 301 can 
be extracted by the less information quantity and by executing the equal process for the moving image data 1 01 and the 
compressed moving image data 301. And, identical moving image can be retrieved and identified from the moving 
image by using the information 305 of the unique identification of the compressed moving image at the higher speed. 
40 [0134] An apparatus and a method for retrieving moving image in the eighth preferred embodiment according to the 
invention will be explained in FIGs. 16, 17A and 17B. 

[0135] FIG. 1 6 is a block diagram showing an apparatus for retrieving moving image according to the present inven- 
tion. Still, an identical code is appended to the thing having the identical composition in FIG. 16 with FIG. 5, FIG. 7 and 
FIG. 13. In FIG. 16, the apparatus for retrieving moving image comprises an extracting unit 901 for extracting informa- 

45 tion of motion compensation in the M x N (M and N are natural numbers) pixels unit of moving image data 101, an 
extracting unit 1301 for calculating an average pixel value, such as information of average color (R, G, B) or information 
of average luminance and average color difference (Y, Cb, Cr), in the M' x N' (M' and N' are natural numbers) pixels unit, 
a detecting unit 501 for detecting a cut point of the moving image data 1 01 as information 502 of a cut point, an extract- 
ing unit 1302 for extracting information 1303 of the feature quantity of compressed moving image, which includes DC 

50 (Direct Current) components of DCT (Discrete Cosine Transform) coefficients of each frame and information of motion 
compensation of each frame, from compressed moving image data 301 provided by the standard compression encod- 
ing process for moving image such as MPEG, a detecting unit 701 for detecting a cut point of the compressed moving 
image data 301 as information 702 of a cut point, an extracting unit 1 603 for arranging at least one information of feature 
quantity from the DC components of the DCT coefficients of each frame and the information of the motion compensa- 

55 tion of each frame or the average pixel value, which are included in the information 1303 extracted by the extracting unit 
901 and the extracting unit 1301 , or the extracting unit 1302, on the time axis and for extracting information 705 of the 
unique identification of compressed moving image to identify the compressed moving image from the arrangement of 
the information of the feature quantity on the time axis in each shot unit of the moving image data 1 01 divided by using 
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the information 502 of the cut point or in each shot unit of the compressed moving image data 301 divided by using the 
information 702 of the cut point, a storing unit 306 for storing the information 705 of the unique identification of the com- 
pressed moving image extracted by the extracting unit 1 603, a retrieving unit 309 for retrieving identical information with 
the information 705 of the unique identification of the compressed moving image stored in the storing unit 306 from 
information 708 of the unique identification of compressed moving image beforehand extracted from other moving 
image data, a storing unit 11 0 for storing the retrieving result provided by the retrieving unit 309, and an output unit 1 1 1 
for displaying and / or printing the retrieving result stored in the storing unit 1 1 0. On the apparatus for retrieving the mov- 
ing image according to the present invention constituted like the above, an operation will be explained in the following. 
[0136] FIGs. 1 7A and 1 7B are flowcharts of the process in the apparatus for retrieving the moving image according 
to the present invention described in FIG. 1 6. Still, an identical step number is appended to the step having the identical 
process in FIGs. 17A and 17B with FIGs. 6A, 6B, 8A, 8B, 15A and 15B. In FIGs. 16, 17A and 17B, at first whether input- 
ted moving image data is compressed is judged (in the step 1 001 ). 

[0137] In case the inputted moving image data is the uncompressed moving image data 1 01 , the extracting unit 901 
for extracting the information of the motion compensation, the extracting unit 1301 for extracting the average value of 
the block unit and the detecting unit 501 for detecting the cut point fetch the moving image data 1 01 to be a retrieval key 
for identifying the moving image (in the step 1701). 

[0138] The extracting unit 901 divides each frame of the moving image data 1 01 into the MxN(M and N are natural 
numbers) pixels units, then calculates the information of the motion compensation in the M x N pixels unit (in the step 
1003). And, the extracting unit 1301 divides each frame of the moving image data 101 into the M' x N' (M' and N' are 
natural numbers) pixels units, then calculates the average pixel value in the M' x N' pixels unit (in the step 1502). And 
more, the detecting unit 501 detects the cut point of the moving image data 1 01 as the information 502 of the cut point 
(in the step 602). 

[0139] The information of the motion compensation in the MxN pixels unit provided by the extracting unit 901 and 
the average pixel value in the M' x NT pixels unit provided by the extracting unit 1 301 are sent to the extracting unit 1 603 
for extracting the information of the unique identification of the compressed moving image with the information 502 of 
the cut point detected by the detecting unit 501 as the information 1303 of the feature quantity of the compressed mov- 
ing image. 

[0140] On the other hand, in case the inputted moving image data is the compressed moving image data 301, the 
extracting unit 1302 for extracting the information of the feature quantity of the compressed moving image and the 
detecting unit 701 for detecting the cut point fetch the compressed moving image data 301 to be a retrieval key for iden- 
tifying the compressed moving image (in the step 801). And the extracting unit 1302 extracts the information 1303 of 
the feature quantity of the compressed moving image, which includes the DC components of the DCT coefficients of 
each frame and the information of the motion compensation of each frame, from the compressed moving image data 
301 (in the step 1 503). And then, the detecting unit 701 detects the cut point of the compressed moving image data 301 
as the information of the cut point 702 (in the step 802). 

[0141] Next, the extracting unit 1603 for extracting the information of the unique identification of the compressed 
moving image calculates values of physical information of the feature quantity of the compressed moving image from 
the DC components of the DCT coefficients of each frame and / or values of the average pixel value of each frame, 
which are included in the information 1 303, mean value of the values, total value of the values and / or difference value 
of the values. On the other hand, the extracting unit 1603 calculates values such as a motion vector, a mean motion 
vector between frames before and behind, a total motion vector, a difference vector and / or a motion vector on a whole 
frame from the information of the motion compensation. And the extracting unit 1603 arranges at least one value from 
the above mentioned values on the time axis and then picks out all values arranged on the time axis, or some values 
from the values arranged on the time axis at uniform intervals or irregular intervals. And then, the extracting unit 1603 
extracts the picked out values in each shot unit of the moving image data 101 divided by using the information 502 of 
the cut point or in each shot unit of the compressed moving image data 301 divided by using the information 702 of the 
cut point as the information 705 of the unique identification of the compressed moving image to be used for identifying 
the compressed moving image (in the step 1702). 

[0142] The storing unit 306 for storing the information of the unique identification of the compressed moving image 
stores the information 705 of the unique identification of the compressed moving image extracted by the extracting unit 
1603 (in the step 804). 

[0143] On the other hand, the information 708 of the unique identification of the compressed moving image is 
extracted from other compressed moving image data (the object compressed moving image data for retrieving) (in the 
step 405). This extracting process can be executed by the processing equal to the foregoing steps 1001 to 804. 
[0144] Then, the retrieving unit 309 for retrieving the information of the unique identification of the compressed 
moving image compares the information 705 of the unique identification of the compressed moving image stored in the 
storing unit 306 with the information 708 of the unique identification of the compressed moving image extracted from 
the other moving image data (in the step 406). 
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[0145] This comparing result is stored in the storing unit 1 1 0 for storing the retrieving result (in the step 207). In the 
case of displaying and / or printing the comparing result (in the step 208), it is outputted by the output unit 111 for dis- 
playing and / or printing the retrieving result (in the step 209). 

[0146] The process of the above mentioned steps 406 to 209 is executed for all of the information 708 of the unique 
5 identification of the compressed moving image extracted from the other moving image data (in the step 410). 

[0147] The apparatus and the method for retrieving the moving image according to the invention are explained the 
above. The M x N pixels unit and the M' x N' pixels unit may be made to be a value equal to the pixel unit of the standard 
compression encoding system such as MPEG. 

[0148] As mentioned above, according to the eighth embodiment of the invention, by utilizing the time concept of 
10 the shot according to the information of the cut point and by executing the preprocessing on the moving image, infor- 
mation 1303 of the unique identification of the compressed moving image for retrieving moving image depends on the 
content of the moving image data 101 and the compressed moving image data 301 can be extracted by the less infor- 
mation quantity and by executing the equal process for the moving image data 101 and the compressed moving image 
data 301. And, identical moving image can be retrieved and identified from the moving image by using the information 
15 1303 of the unique identification of the compressed moving image at the higher speed. 

[0149] An apparatus and a method for retrieving moving image in the ninth preferred embodiment according to the 
invention will be explained in FIGs. 18, 1 9A and 1 9B. 

[0150] FIG. 18 is a block diagram showing an apparatus for retrieving moving image according to the present inven- 
tion. Still, an identical code is appended to the thing having the identical composition in FIG. 18 with FIG. 3 and FIG. 9. 

20 In FIG. 18, the apparatus for retrieving moving image comprises a DCT (Discrete Cosine Transform) operation unit 902 
for obtaining a DCT coefficient by executing a DCT operation in the M x N (M and N are natural numbers) pixels unit of 
moving image data 101 as information 303 of the feature quantity of compressed moving image, an extracting unit 302 
for extracting information 303 of the feature quantity of compressed moving image, which includes the DCT coefficients 
of each frame, from compressed moving image data 301 provided by the standard compression encoding process for 

25 moving image such as MPEG, an extracting unit 304 for arranging at least one information of feature quantity from the 
DCT coefficients of each frame, which is included in the information 303 extracted by the DCT operation unit 902 or by 
the extracting unit 302, on the time axis and for extracting information 305 of the unique identification of compressed 
moving image to identify the compressed moving image from the arrangement of the information of feature quantity on 
the time axis, a storing unit 306 for storing the information 305 of the unique identification of the compressed moving 

30 image extracted by the extracting unit 304, a retrieving unit 309 for retrieving identical information with the information 
305 of the unique identification of the compressed moving image stored in the storing unit 306 from information 308 of 
the unique identification of compressed moving image beforehand extracted from other moving image data, a storing 
unit 1 10 for storing the retrieving result provided by the retrieving unit 309, and an output unit 111 for displaying and / 
or printing the retrieving result stored in the storing unit 1 1 0. On the apparatus for retrieving the moving image according 

35 to the present invention constituted like the above, an operation will be explained in the following. 

[0151] FIGs. 1 9A and 1 9B are flowcharts of the process in the apparatus for retrieving the moving image according 
to the present invention described in FIG. 1 8. Still, an identical step number is appended to the step having the identical 
process in FIGs. 19A and 19B with FIGs. 10A and 10B. In FIGs. 18, 19A and 19B, at first whether inputted moving 
image data is compressed is judged, (in the step 1001). 

40 [0152] In case the inputted moving image data is the uncompressed moving image data 101, the DCT operation 
unit 902 fetches the moving image data 101 to be a retrieval key for identifying the moving image (in the step 1901). 
[0153] The DCT operation unit 902 divides each frame of the moving image data 101 into the M x N (M and N are 
natural numbers) pixels units, then calculates the DCT coefficient by executing the DCT operation in the M x N pixels 
unit as the information 303 of the feature quantity of the compressed moving image (in the step 1902). 

45 [0154] On the other hand, in case the inputted moving image data is the compressed moving image data 301 , the 
extracting unit 302 for extracting the information of the feature quantity of the compressed moving image fetches the 
compressed moving image data 301 to be a retrieval key for identifying the compressed moving image (in the step 401 ). 
And the extracting unit 302 extracts the information 303 of the feature quantity of the compressed moving image, which 
includes the DCT coefficients of each frame, from the compressed moving image data 301 as (in the step 1903). 

50 [0155] Next, the extracting unit 304 for extracting the information of the unique identification of the compressed 
moving image calculates values of physical information of the feature quantity of the compressed moving image from 
the DCT coefficients of each frame, which is included in the information 303 extracted by the extracting unit 902, mean 
value of the values, total value of the values and / or difference value of the values. And the extracting unit 304 arranges 
at least one value from the above mentioned values on the time axis and then picks out all values arranged on the time 

55 axis, or some values from the values arranged on the time axis at uniform intervals or irregular intervals. And then, the 
extracting unit 304 extracts the picked out values as the information 305 of the unique identification of the compressed 
moving image to be used for identifying the compressed moving image (in the step 1904). 

[0156] The storing unit 306 for storing the information of the unique identification of the compressed moving image 
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stores the information 305 of the unique identification of the compressed moving image extracted by the extracting unit 
304 (in the step 404). 

[0157] On the other hand, the information 308 of the unique identification of the compressed moving image is 
extracted from other compressed moving image data (the object compressed moving image data for retrieving) (in the 
step 405). This extracting process can be executed by the processing equal to the foregoing steps 1001 to 404. 
[0158] Then, the retrieving unit 309 for retrieving the information of the unique identification of the compressed 
moving image compares the information 305 of the unique identification of the compressed moving image stored in the 
storing unit 306 with the information 308 of the unique identification of the compressed moving image extracted from 
the other moving image data (in the step 406). 

[0159] This comparing result is stored in the storing unit 1 1 0 for storing the retrieving result (in the step 207). In the 
case of displaying and / or printing the comparing result (in the step 208), it is outputted by the output unit 111 for dis- 
playing and / or printing the retrieving result (in the step 209). 

[0160] The process of the above mentioned steps 406 to 209 is executed for all of the information 308 of the unique 
identification of the compressed moving image extracted from the other moving image data (in the step 410). 
[0161] The apparatus and the method for retrieving the moving image according to the invention are explained the 
above. The M x N pixels unit may be made to be a value equal to the pixel unit of the standard compression encoding 
system such as MPEG. 

[0162] As mentioned above, according to the ninth embodiment of the invention, by executing the preprocessing on 
the moving image 101, the information 305 of the unique identification of the compressed moving image for retrieving 
moving image depends on the content of the moving image data 1 01 and the compressed moving image data 301 can 
be extracted by executing the equal process. And, identical moving image can be retrieved and identified from the mov- 
ing image by using the information 305 of the unique identification of the compressed moving image at the higher 
speed. 

[0163] An apparatus and a method for retrieving moving image in the tenth preferred embodiment according to the 
invention will be explained in FIGs. 20, 21 A and 21 B. 

[0164] FIG. 20 is a block diagram showing an apparatus for retrieving moving image according to the present inven- 
tion. Still, an identical code is appended to the thing having the identical composition in FIG. 20 with FIG. 11. In FIG. 
20, the apparatus for retrieving moving image comprises a DCT (Discrete Cosine Transform) operation unit 902 for 
obtaining a DCT coefficient as information 303 of the feature quantity of compressed moving image by executing a DCT 
operation in the M x N (M and N are natural numbers) pixels unit of moving image data 101, a detecting unit 501 for 
detecting a cut point of the moving image data 1 01 as information 502 of a cut point, an extracting unit 302 for extracting 
information 303 of the feature quantity of compressed moving image, which includes DCT coefficients of each frame, 
from compressed moving image data 301 provided by the standard compression encoding process for moving image 
such as MPEG, a detecting unit 701 for detecting a cut point of the compressed moving image data 301 as information 
702 of a cut point, an extracting unit 703 for arranging at least one information of feature quantity from the DCT coeffi- 
cients of each frame, which is included in the information 303 extracted by the DCT operation unit 902 or the extracting 
unit 302, on the time axis and for extracting information 705 of the unique identification of compressed moving image 
to identify the compressed moving image from the arrangement of the information of the feature quantity on the time 
axis in each shot unit of the compressed moving image data 301 divided by using the information 502 of the cut point 
and / or the information 702 of the cut point, a storing unit 306 for storing the information 705 of the unique identification 
of the compressed moving image extracted by the extracting unit 703, a retrieving unit 309 for retrieving identical infor- 
mation with the information 705 of the unique identification of the compressed moving image stored in the storing unit 
306 from information 708 of the unique identification of compressed moving image beforehand extracted from other 
moving image data, a storing unit 1 10 for storing the retrieving result provided by the retrieving unit 309, and an output 
unit 111 for displaying and / or printing the retrieving result stored in the storing unit 1 1 0. On the apparatus for retrieving 
the moving image according to the present invention constituted like the above, an operation will be explained in the 
following. 

[0165] FIGs. 21 A and 21 B are flowcharts of the process in the apparatus for retrieving the moving image according 
to the present invention described in FIG. 20. Still, an identical step number is appended to the step having the identical 
process in FIGs. 21 A and 21 B with FIGs. 12A, 12B, 19A and 19B. In FIGs. 20, 21 A and 21 B, at first whether inputted 
moving image data is compressed is judged, (in the step 1 001 ). 

[0166] In case the inputted moving image data is the uncompressed moving image data 101, the DCT operation 
unit 902 and the detecting unit 501 for detecting the cut point fetch the moving image data 1 01 to be a retrieval key for 
identifying the moving image (in the step 2101). 

[0167] The DCT operation unit 902 divides each frame of the moving image data 101 into the M x N (M and N are 
natural numbers) pixels units, then calculates the DCT coefficient by executing the DCT operation in the M x N pixels 
unit as the information 303 of the feature quantity of the compressed moving image (in the step 1902). On the other 
hand, the detecting unit 501 for detecting the cut point detects the cut point of the moving image data 1 01 as the infor- 
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mation 502 of the cut point (in the step 602). 

[0168] The DCT coefficient in the M x N pixels unit provides by the DCT operation unit 902 is sent to the extracting 
unit 703 for extracting the information of the unique identification of the compressed moving image with the information 
502 detected by the detecting unit 501 as the information 303 of the feature quantity of the compressed moving image. 
[0169] On the other hand, in case the inputted moving image data is the compressed moving image data 301, the 
extracting unit 302 for extracting the information of the feature quantity of the compressed moving image and the detect- 
ing unit 701 for detecting the cut point fetch the compressed moving image data 301 to be a retrieval key for identifying 
the compressed moving image (in the step 801 ). Next, the extracting unit 302 extracts the information 303 of the feature 
quantity of the compressed moving image, which includes the DCT coefficients of each frame, from the compressed 
moving image data 301 (in the step 1903). And, the detecting unit 701 detects the cut point of the compressed moving 
image data 301 as the information 702 of the cut point (in the step 802). 

[0170] Next, the extracting unit 703 for extracting the information of the unique identification of the compressed 
moving image calculates values of physical information of the feature quantity of the compressed moving image from 
the DCT coefficients of each frame, which is included in the information 303, mean value of the values, total value of 
the values and / or difference value of the values. And the extracting unit 703 arranges at least one value from the above 
mentioned values on the time axis and then picks out all values arranged on the time axis, or some values from the val- 
ues arranged on the time axis at uniform intervals or irregular intervals. And then, the extracting unit 703 extracts the 
picked out values in each shot unit of the moving image data 101 divided by using the information 502 or in each shot 
unit of the compressed moving image data 301 divided by using the information 702 as the information 705 of the 
unique identification of the compressed moving image to be used for identifying the compressed moving image (in the 
step 2102). 

[0171] The storing unit 306 for storing the information of the unique identification of the compressed moving image 
stores the information 705 of the unique identification of the compressed moving image extracted by the extracting unit 
703 (in the step 804). 

[0172] On the other hand, the information 708 of the unique identification of the compressed moving image is 
extracted from other compressed moving image data (the object compressed moving image data for retrieving) (in the 
step 405). This extracting process can be executed by the processing equal to the foregoing steps 1001 to 804. 
[0173] Then, the retrieving unit 309 for retrieving the information of the unique identification of the compressed 
moving image compares the information 705 of the unique identification of the compressed moving image stored in the 
storing unit 306 with the information 708 of the unique identification of the compressed moving image extracted from 
the other moving image data (in the step 406). 

[0174] This comparing result is stored in the storing unit 1 1 0 for storing the retrieving result (in the step 207). In the 
case of displaying and / or printing the comparing result (in the step 208), it is outputted by the output unit 111 for dis- 
playing and / or printing the retrieving result (in the step 209). 

[0175] The process of the above mentioned steps 406 to 209 is executed for all of the information 708 of the unique 
identification of the compressed moving image extracted from the other moving image data (in the step 410). 
[0176] The apparatus and the method for retrieving the moving image according to the invention are explained the 
above. The M x N pixels unit may be made to be a value equal to the pixel unit of the standard compression encoding 
system such as MPEG. 

[0177] As mentioned above, according to the tenth embodiment of the invention, by executing the preprocessing on 
the uncompressed moving image data 101, the information 705 of the unique identification of the compressed moving 
image for retrieving moving image depends on the content of the moving image 101 and the compressed moving image 
301 can be extracted by the equal processing. And, the information 705 for retrieving moving image can be extracted 
by the less information quantity with utilizing the time concept of the shot according to the information 502 and the infor- 
mation 702. Identical moving image can be retrieved and identified from the moving image by using the information 705 
of the unique identification of the compressed moving image at the higher speed. 

[0178] An apparatus and a method for retrieving moving image in the eleventh preferred embodiment according to 
the invention will be explained in FIGs. 22, 23A and 23B. 

[0179] FIG. 22 is a block diagram showing an apparatus for retrieving moving image according to the present inven- 
tion. Still, an identical code is appended to the thing having the identical composition in FIG. 22 with FIG. 13. In FIG. 
22, the apparatus for retrieving moving image comprises an extracting unit 1301 for calculating an average pixel value, 
such as information of average color (R, G, B) or information of average luminance and average color difference (Y, Cb, 
Cr), in the M x N (M and N are natural numbers) pixels unit, an extracting unit 1302 for extracting information 1303 of 
the feature quantity of compressed moving image, which includes DC (Direct Current) components of DCT (Discrete 
Cosine Transform) coefficients of each frame, from compressed moving image data 301 provided by the standard com- 
pression encoding process for moving image such as MPEG, an extracting unit 1 304 for arranging at least one informa- 
tion of feature quantity from the DC components of the DCT coefficients of each frame or the average pixel value, which 
is included in the information 1303 extracted by the extracting unit 1301 and / or the extracting unit 1302, on the time 
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axis and for extracting information 305 of the unique identification of compressed moving image to identify compressed 
moving image from the arrangement of the information of the feature quantity on the time axis, a storing unit 306 for 
storing the information 305 of the unique identification of the compressed moving image extracted by the extracting unit 
1304, a retrieving unit 309 for retrieving identical information with the information 305 of the unique identification of the 
compressed moving image stored in the storing unit 306 from information 308 of the unique identification of com- 
pressed moving image beforehand extracted from other moving image data, a storing unit 1 10 for storing the retrieving 
result provided by the retrieving unit 309, and an output unit 111 for displaying and / or printing the retrieving result 
stored in the storing unit 1 10. On the apparatus for retrieving the moving image according to the present invention con- 
stituted like the above, an operation will be explained in the following. 

[0180] FIGs. 23A and 23B are flowcharts of the process in the apparatus for retrieving the moving image according 
to the present invention described in FIG. 22. Still, an identical step number is appended to the step having the identical 
process in FIGs. 23A and 23B with FIGs. 15A and 15B. In FIGs. 22, 23A and 23B, at first whether inputted moving 
image data is compressed is judged, (in the step 1001). 

[0181] In case the inputted moving image data is the uncompressed moving image data 101, the extracting unit 
1301 for extracting the average value of the block unit fetches the moving image data 1 01 to be a retrieval key for iden- 
tifying the moving image (in the step 2301). 

[0182] The extracting unit 1301 divides each frame of the moving image data 101 into the MxN(M and N are nat- 
ural numbers) pixels units, then calculates the average pixel value in the MxN pixels unit as the information 1303 of 
the feature quantity of the compressed moving image (in the step 1502). 

[0183] On the other hand, in case the inputted moving image data is the compressed moving image data 301, the 
extracting unit 1302 for extracting the information of the feature quantity of the compressed moving image fetches the 
compressed moving image data 301 to be a retrieval key for identifying the compressed moving image (in the step 401 ). 
Then, the extracting unit 1302 extracts the information 1303 of the feature quantity of the compressed moving image, 
which includes the DC components of the DCT coefficients of each frame, from the compressed moving image data 
301 (in the step 1503). 

[0184] Next, the extracting unit 1304 for extracting the information of the unique identification of the compressed 
moving image calculates values of physical information of the feature quantity of compressed moving image from the 
DC components of the DCT coefficients of each frame and / or values of the average pixel value of each frame, which 
are included in the information 1 303, mean value of the values, total value of the values and / or difference value of the 
values. And the extracting unit 1 304 arranges at least one value from the above mentioned values on the time axis and 
then picks out all values arranged on the time axis, or some values from the values arranged on the time axis at uniform 
intervals or irregular intervals. And then, the extracting unit 1304 extracts the picked out values as the information 305 
of the unique identification of the compressed moving image to be used for identifying the compressed moving image 
(in the step 1504). 

[0185] The storing unit 306 for storing the information of the unique identification of the compressed moving image 
stores the information 305 of the unique identification of the compressed moving image extracted by the extracting unit 
1304 (in the step 404). 

[0186] On the other hand, the information 308 of the unique identification of the compressed moving image is 
extracted from other compressed moving image data (the object compressed moving image data for retrieving) (in the 
step 405). This extracting process can be executed by the processing equal to the foregoing steps 1001 to 404. 
[0187] Then, the retrieving unit 309 for retrieving the information of the unique identification of the compressed 
moving image compares the information 305 of the unique identification of the compressed moving image stored in the 
storing unit 306 with the information 308 of the unique identification of the compressed moving image extracted from 
the other moving image data (in the step 406). 

[0188] This comparing result is stored in the storing unit 1 1 0 for storing the retrieving result (in the step 207). In the 
case of displaying and / or printing the comparing result (in the step 208), it is outputted by the output unit 111 for dis- 
playing and / or printing the retrieving result (in the step 209). 

[0189] The process of the above mentioned steps 406 to 209 is executed for all of the information 308 of the unique 
identification of the compressed moving image extracted from the other moving image data (in the step 410). 
[0190] The apparatus and the method for retrieving the moving image according to the invention are explained the 
above. The above mentioned extracting unit 1304 calculates values of the physical information of the feature quantity 
of the compressed moving image from the DC components of the DCT coefficients of each frame or values of the aver- 
age pixel value of each frame, which are included in the information 1303, mean value of the values, total value of the 
values and / or difference value of the values. However, the extracting unit 1 304 may calculate the above mentioned val- 
ues by using a part or multiple parts of the physical information of the feature quantity of the compressed moving image 
from the DC components of the DCT coefficients of each frame or the average pixel value of each frame, which are 
included in the information 1303. 

[0191] And more, the extracting unit 1304 may execute the weighting on the physical information of the feature 
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quantity of the compressed moving image from the DC components of the DCT coefficients of each frame or the aver- 
age pixel value of each frame, which are included in the information 1303, then may calculate the above mentioned val- 
ues by the weighted information of the feature quantity of the compressed moving image or the weighted average pixel 
value. 

[0192] Moreover, in the extracting unit 1304, values provided by quantizing the above mentioned physical informa- 
tion or average pixel value or the like may be applied. 

[0193] Further more, the M x N pixels unit may be made to be a value equal to the pixel unit of the standard com- 
pression encoding system such as MPEG. 

[0194] As mentioned above, according to the eleventh embodiment of the invention, by executing the preprocessing 
on the moving image, the information 305 of the unique identification of the compressed moving image for retrieving the 
moving image depends on the content of the moving image data 1 01 and the compressed moving image data 301 can 
be extracted by the less information quantity and by executing the equal process for the moving image data 1 01 and the 
compressed moving image data 301. And, identical moving image can be retrieved and identified from the moving 
image by using the information 305 of the unique identification of the compressed moving image at the higher speed. 
[0195] An apparatus and a method for retrieving moving image in the twelfth preferred embodiment according to 
the invention will be explained in FIGs. 24, 25A and 25B. 

[0196] FIG. 24 is a block diagram showing an apparatus for retrieving moving image according to the present inven- 
tion. Still, an identical code is appended to the thing having the identical composition in FIG. 24 with FIG. 16. In FIG. 
24, the apparatus for retrieving moving image comprises an extracting unit 1301 for calculating an average pixel value, 
such as information of average color (R, G, B) or information of average luminance and average color difference (Y, Cb, 
Cr), in the M x N (M and N are natural numbers) pixels unit of moving image data 1 01 as information 1 303 of the feature 
quantity of compressed moving image, a detecting unit 501 for detecting a cut point of the moving image data 101 as 
information 502 of a cut point, an extracting unit 1302 for extracting information 1303 of the feature quantity of com- 
pressed moving image, which includes DC (Direct Current) components of DCT (Discrete Cosine Transform) coeffi- 
cients of each frame, from compressed moving image data 301 provided by the standard compression encoding 
process for moving image such as MPEG, a detecting unit 701 for detecting a cut point of the compressed moving 
image data 301 as information 702 of a cut point, an extracting unit 1 603 for arranging at least one information of feature 
quantity from the DC components of the DCT coefficients of each frame or the average pixel value, which is included in 
the information 1303 extracted by the extracting unit 1301 and / or the extracting unit 1302, on the time axis and for 
extracting information 705 of the unique identification of compressed moving image to identify the compressed moving 
image from the arrangement of the information of the feature quantity on the time axis in each shot unit of the moving 
image data 101 divided by using the information 502 of the cut point or in each shot unit of the compressed moving 
image data 301 divided by using the information 702 of the cut point, a storing unit 306 for storing the information 705 
of the unique identification of the compressed moving image extracted by the extracting unit 1603, a retrieving unit 309 
for retrieving identical information with the information 705 of the unique identification of the compressed moving image 
stored in the storing unit 306 from information 708 of the unique identification of compressed moving image beforehand 
extracted from other moving image data, a storing unit 1 10 for storing the retrieving result provided by the retrieving unit 
309, and an output unit 111 for displaying and / or printing the retrieving result stored in the storing unit 110. On the 
apparatus for retrieving the moving image according to the present invention constituted like the above, an operation 
will be explained in the following. 

[0197] FIGs. 25A and 25B are flowcharts of the process in the apparatus for retrieving the moving image according 
to the present invention described in FIG. 24. Still, an identical step number is appended to the step having the identical 
process in FIGs. 25A and 25B with FIGs. 17A and 17B. In FIGs. 24, 25A and 25B, at first whether inputted moving 
image data is compressed is judged (in the step 1 001 ). 

[0198] In case the inputted moving image data is the uncompressed moving image data 101, the extracting unit 
1301 for extracting the average value of the block unit and the detecting unit 501 for detecting the cut point fetch the 
moving image data 101 to be a retrieval key for identifying the moving image (in the step 2501). 

[0199] The extracting unit 1301 divides each frame of the moving image data 101 into the M x N (M and N are nat- 
ural numbers) pixels units, then calculates the average pixel value in the M x N pixels unit (in the step 1502). And, the 
detecting unit 501 detects the cut point of the moving image data 1 01 as the information 502 of the cut point (in the step 
602). 

[0200] On the other hand, in case the inputted moving image data is the compressed moving image data 301 , the 
extracting unit 1302 for extracting the information of the feature quantity of the compressed moving image and the 
detecting unit 701 for detecting the cut point fetch the compressed moving image data 301 to be a retrieval key for iden- 
tifying the compressed moving image (in the step 801). Then, the extracting unit 1302 extracts the information 1303 of 
the feature quantity of the compressed moving image, which includes the DC components of the DCT coefficients of 
each frame, from the compressed moving image data 301 (in the step 1503). And, the detecting unit 701 detects the 
cut point of the compressed moving image data 301 as the information 702 of the cut point (in the step 802). 
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[0201] Next, the extracting unit 1603 for extracting the information of the unique identification of the compressed 
moving image calculates values of physical information of the feature quantity of compressed moving image from the 
DC components of the DCT coefficients of each frame and / or the average pixel value of each frame, which are 
included in the information 1303, mean value of the values, total value of the values and / or difference value of the val- 

5 ues. And the extracting unit 1603 arranges at least one value from the above mentioned values on the time axis and 
then picks out all values arranged on the time axis, or some values from the values arranged on the time axis at uniform 
intervals or irregular intervals. And then, the extracting unit 1603 extracts the picked out values in each shot unit of the 
moving image data 101 divided by using the information 502 of the cut point or in each shot unit of the compressed 
moving image data 301 divided by using the information 702 of the cut point as the information 705 of the unique iden- 

10 tification of the compressed moving image to be used for identifying the compressed moving image (in the step 1702). 
[0202] The storing unit 306 for storing the information of the unique identification of the compressed moving image 
stores the information 705 of the unique identification of the compressed moving image extracted by the extracting unit 
1603 (in the step 804). 

[0203] On the other hand, the information 708 of the unique identification of the compressed moving image is 
15 extracted from other compressed moving image data (the object compressed moving image data for retrieving) (in the 
step 405). This extracting process can be executed by the processing equal to the foregoing steps 1001 to 804. 
[0204] Then, the retrieving unit 309 for retrieving the information of the unique identification of the compressed 
moving image compares the information 705 of the unique identification of the compressed moving image stored in the 
storing unit 306 with the information 708 of the unique identification of the compressed moving image extracted from 
20 the other moving image data (in the step 406). 

[0205] This comparing result is stored in the storing unit 1 1 0 for storing the retrieving result (in the step 207). In the 
case of displaying and / or printing the comparing result (in the step 208), it is outputted by the output unit 111 for dis- 
playing and / or printing the retrieving result (in the step 209). 

[0206] The process of the above mentioned steps 406 to 209 is executed for all of the information 708 of the unique 
25 identification of the compressed moving image extracted from the other moving image data (in the step 410). 

[0207] The apparatus and the method for retrieving the moving image according to the invention are explained the 
above. The M x N pixels unit may be made to be a value equal to the pixel unit of the standard compression encoding 
system such as MPEG. 

[0208] As mentioned above, according to the twelfth embodiment of the invention, by utilizing the time concept of 
30 the shot according to the information of the cut point and executing the simple preprocessing on the moving image, the 
information 1303 of the unique identification of the compressed moving image for retrieving the moving image depends 
on the content of the moving image data 1 01 and the compressed moving image data 301 can be extracted by the less 
information quantity and by executing the equal process for the moving image data 101 and the compressed moving 
image data 301 . And, identical moving image can be retrieved and identified from the moving image by using the infor- 
35 mation 1303 of the unique identification of the compressed moving image at the higher speed. 

[0209] An apparatus and a method for retrieving moving image in the thirteenth preferred embodiment according to 
the invention will be explained in FIGs. 26, 27A and 27B. 

[0210] FIG. 26 is a block diagram showing an apparatus for retrieving moving image according to the present inven- 
tion. Still, an identical code is appended to the thing having the identical composition in FIG. 26 with FIG. 1 and FIG. 3. 

40 In FIG. 26, the apparatus for retrieving moving image comprises an extracting unit 1 02 for extracting information 1 03 of 
the feature quantity of moving image, which includes physical information of the feature quantity of moving image such 
as information of luminance and color difference (Y, Cb, Cr) of each frame and / or information of color (R, G, B) of each 
frame, from moving image data 101, an extracting unit 104 for arranging at least one physical information included in 
the information 103 (such as the information of the luminance and the color difference (Y, Cb, Cr) of each frame and / 

45 or the information of the color (R, G, B) of each frame) on the time axis and for extracting information 1 05 of the unique 
identification of moving image to identify the moving image from the arrangement of the physical information on the time 
axis, a storing unit 106 for storing the information 105 of the unique identification of the moving image extracted by the 
extracting unit 104, a retrieving unit 109 for retrieving identical information with the information 105 of the unique iden- 
tification of the moving image stored in the storing unit 106 from information 108 of the unique identification of moving 

50 image beforehand extracted from other moving image data, an extracting unit 302 for extracting information 303 of the 
feature quantity of compressed moving image, which includes DCT (Discrete Cosine Transform) coefficients of each 
frame and information of motion compensation of each frame, from compressed moving image data 301 provided by 
the standard compression encoding process for moving image such as MPEG, an extracting unit 304 for arranging at 
least one information of feature quantity from the DCT coefficients of each frame and the information of the motion com- 

55 pensation of each frame, which are included in the information 303, on the time axis and for extracting information 305 
of the unique identification of compressed moving image to identify the compressed moving image from the arrange- 
ment of the information of the feature quantity on the time axis, a storing unit 306 for storing the information 305 of the 
unique identification of the compressed moving image extracted by the extracting unit 304, a retrieving unit 309 for 
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retrieving identical information with the information 305 of the unique identification of the compressed moving image 
stored in the storing unit 306 from information 308 of the unique identification of compressed moving image beforehand 
extracted from other moving image data, a judging unit 2602 forjudging whether inputted digital moving image data 
2601 is the moving image data 101 or the compressed moving image data 301 and for outputting the moving image 
data 101 to the extracting unit 1 02 and the compressed moving image data 301 to the extracting unit 302, a storing unit 
1 10 for storing a retrieving result provided by the retrieving unit 109 and / or by the retrieving unit 309, and an output 
unit 111 for displaying and / or printing the retrieving result stored in the storing unit 1 1 0. 

[021 1] FIGs. 27A and 27B are flowcharts of the process in the apparatus for retrieving the moving image according 
to the present invention described in FIG. 26. Still, an identical step number is appended to the step having the identical 
process in FIGs. 27A and 27B with FIGs. 2A, 2B, 4A and 4B. In FIGs. 26, 27A and 27B, at first the compressed / uncom- 
pressed judging unit 2602 acquires the digital moving image data 2601 (in the step 2701). Then, the compressed / 
uncompressed judging unit 2602 judges whether the acquired digital moving image data 2601 is the uncompressed 
moving image data 101 or the compressed moving image data 301 (in the step 2702). 

[021 2] In case the digital moving image data 2601 is the uncompressed moving image data 1 01 , the following proc- 
ess (the steps 202 to 210) is executed. 

[0213] The extracting unit 102 extracts the information 103 of the feature quantity of the moving image, which 
includes the physical information of the feature quantity of the moving image such as the information of the luminance 
and the color difference (Y, Cb, Cr) of each frame and / or the information of the color (R, G, B) of each frame, from the 
moving image data 101 (in the step 202). 

[0214] Next, the extracting unit 104 for extracting the information of the unique identification of the moving image 
arranges at least one value from values of the physical information of the feature quantity of the moving image (such as 
the information of the luminance and the color difference (Y, Cb, Cr) of each frame and / or the information of the color 
(R, G, B) of each frame) based on the information 103, mean value of the values, total value of the values and / or dif- 
ference value of the values on the time axis. Then, the extracting unit 1 04 picks out all values arranged on the time axis, 
or some values from the values arranged on the time axis at uniform intervals or irregular intervals. And then, the 
extracting unit 1 04 extracts the picked out values as the information 1 05 of the unique identification of the moving image 
to be used for identifying the moving image (in the step 203). 

[0215] The storing unit 106 for storing the information of the unique identification of the moving image stores the 
information 105 of the unique identification of the moving image extracted by the extracting unit 104 (in the step 204). 
[0216] On the other hand, the information 108 of the unique identification of the moving image is extracted from 
other moving image data (the object moving image data for retrieving) (in the step 205). This extracting process can be 
executed by the processing equal to the foregoing steps 201 to 204 (in FIGs. 2A and 2B). 

[0217] Then, the retrieving unit 109 for retrieving the information of the unique identification of the moving image 
compares the information 105 of the unique identification of the moving image stored in the storing unit 106 with the 
information 1 08 of the unique identification of the moving image extracted from the other moving image data (in the step 
206). 

[021 8] This comparing result is stored in the storing unit 1 1 0 for storing the retrieving result (in the step 207). In the 
case of displaying and / or printing the comparing result (in the step 208), it is outputted by the output unit 111 for dis- 
playing and / or printing the retrieving result (in the step 209). 

[0219] The process of the above mentioned steps 206 to 209 is executed for all of the information 1 08 of the unique 
identification of the moving image extracted from the other moving image data (in the step 210). 

[0220] In case the digital moving image data 2601 is the compressed moving image data 301 , the following process 
(the steps 402 to 410) is executed. 

[0221] The extracting unit 302 extracts the information 303 of the feature quantity of the compressed moving image, 
which includes the DCT coefficients of each frame and the information of the motion compensation of each frame, from 
the compressed moving image data 301 (in the step 402). 

[0222] Next, the extracting unit 304 for extracting the information of the unique identification of the compressed 
moving image calculates values of physical information of the feature quantity of the compressed moving image from 
the DCT coefficients of each frame, which is included in the information 303, mean value of the values, total value of 
the values and / or difference value of the values. On the other hand, the extracting unit 304 calculates values such as 
a motion vector, a mean motion vector between frames before and behind, a total motion vector, a difference vector and 
/ or a motion vector on a whole frame from the information of the motion compensation. And the extracting unit 304 
arranges at least one value from the above mentioned values on the time axis and then picks out all values arranged 
on the time axis, or some values from the values arranged on the time axis at uniform intervals or irregular intervals. 
And then, the extracting unit 304 extracts the picked out values as the information 305 of the unique identification of the 
compressed moving image to be used for identifying the compressed moving image (in the step 403). 
[0223] The storing unit 306 for storing the information of the unique identification of the compressed moving image 
stores the information 305 of the unique identification of the compressed moving image extracted by the extracting unit 
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304 (in the step 404). 

[0224] On the other hand, the information 308 of the unique identification of the compressed moving image is 
extracted from other compressed moving image data (the object compressed moving image data for retrieving) (in the 
step 405). This extracting process can be executed by the processing equal to the foregoing steps 401 to 404 (in FIGs. 
4A and 4B). 

[0225] Then, the retrieving unit 309 for retrieving the information of the unique identification of the compressed 
moving image compares the information 305 of the unique identification of the compressed moving image stored in the 
storing unit 306 with the information 308 of the unique identification of the compressed moving image extracted from 
the other moving image data (in the step 406). 

[0226] This comparing result is stored in the storing unit 11 0 for storing the retrieving result (in the step 207). In the 
case of displaying and / or printing the comparing result (in the step 208), it is outputted by the output unit 111 for dis- 
playing and / or printing the retrieving result (in the step 209). 

[0227] The process of the above mentioned steps 406 to 209 is executed for all of the information 308 of the unique 
identification of the compressed moving image extracted from the other moving image data (in the step 410). 
[0228] As mentioned above, according to the thirteenth embodiment of the invention, whether the inputted digital 
moving image data 2601 is the uncompressed moving image data 101 or the compressed moving image data 301 can 
be judged automatically. Consequently, identical moving image can be retrieved and identified from the moving image 
at the higher speed regardless of the type (compressed / uncompressed) of image data to be retrieved. 
[0229] An apparatus and a method for retrieving moving image in the fourteenth preferred embodiment according 
to the invention will be explained in FIGs. 28, 29A and 29B. 

[0230] FIG. 28 is a block diagram showing an apparatus for retrieving moving image according to the present inven- 
tion. Still, an identical code is appended to the thing having the identical composition in FIG. 28 with FIG. 5 and FIG. 7. 
In FIG. 28, the apparatus for retrieving moving image comprises an extracting unit 102 for extracting information 103 of 
the feature quantity of moving image, which includes physical information of the feature quantity of moving image such 
as information of luminance and color difference (Y, Cb, Cr) of each frame and / or information of color (R, G, B) of each 
frame, from moving image data 1 01 , a detecting unit 501 for detecting a cut point of the moving image data 101 as infor- 
mation 502 of a cut point, an extracting unit 503 for arranging at least one physical information included in the informa- 
tion 103 (such as the information of the luminance and the color difference (Y, Cb, Cr) of each frame and / or the 
information of the color (R, G, B) of each frame) on the time axis and for extracting information 505 of the unique iden- 
tification of moving image to identify the moving image in each shot unit of the moving image data 1 01 divided by using 
the information 502 of the cut point from the arrangement of the physical information on the time axis, a storing unit 1 06 
for storing the information 505 of the unique identification of the moving image extracted by the extracting unit 503, a 
retrieving unit 109 for retrieving identical information with the information 505 of the unique identification of the moving 
image stored in the storing unit 106 from information 508 of the unique identification of moving image beforehand 
extracted from other moving image data, an extracting unit 302 for extracting information 303 of the feature quantity of 
compressed moving image, which includes DCT (Discrete Cosine Transform) coefficients of each frame and informa- 
tion of motion compensation of each frame, from compressed moving image data 301 provided by the standard com- 
pression encoding process for moving image such as MPEG, a detecting unit 701 for detecting a cut point of the 
compressed moving image data 301 as information 702 of a cut point, an extracting unit 703 for arranging at least one 
information of feature quantity from the DCT coefficients of each frame and the information of the motion compensation 
of each frame, which are included in the information 303, on the time axis and for extracting information 705 of the 
unique identification of compressed moving image to identify the compressed moving image from the arrangement of 
the information of the feature quantity on the time axis in each shot unit of the compressed moving image data 301 
divided by using the information 702 of the cut point, a storing unit 306 for storing the information 705 of the unique iden- 
tification of the compressed moving image extracted by the extracting unit 703, a retrieving unit 309 for retrieving iden- 
tical information with the information 705 of the unique identification of the compressed moving image stored in the 
storing unit 306 from information 708 of the unique identification of compressed moving image beforehand extracted 
from other moving image data, a judging unit 2602 for judging whether inputted digital moving image data 2601 is the 
moving image data 101 or the compressed moving image data 301 and for outputting the moving image data 1 01 to the 
extracting unit 1 02 and the compressed moving image data 301 to the extracting unit 302, a storing unit 1 1 0 for storing 
the retrieving result provided by the retrieving unit 109 and / or the retrieving unit 309, and an output unit 111 for dis- 
playing and / or printing the retrieving result stored in the storing unit 1 1 0. 

[0231] FIGs. 29A and 29B are flowcharts of the process in the apparatus for retrieving the moving image according 
to the present invention described in FIG. 28. Still, an identical step number is appended to the step having the identical 
process in FIGs. 29A and 29B with FIGs. 6A, 6B, 8A and 8B. In FIGs. 28, 29A and 29B, at first the compressed / uncom- 
pressed judging unit 2602 acquires the digital moving image data 2601 (in the step 2701). Then, the compressed / 
uncompressed judging unit 2602 judges whether the acquired digital moving image data 2601 is the uncompressed 
moving image data 101 or the compressed moving image data 301 (in the step 2702). 
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[0232] In case the digital moving image data 2601 is the uncompressed moving image data 1 01 , the following proc- 
ess (the steps 202 to 210) is executed. 

[0233] The extracting unit 102 extracts the information 103 of the feature quantity of the moving image, which 
includes the physical information of the feature quantity of the moving image such as the information of the luminance 
and the color difference (Y, Cb, Cr) of each frame and / or the information of the color (R, G, B) of each frame, from the 
moving image data 101 (in the step 202). 

[0234] And, the detecting unit 501 detects the cut point of the moving image data 1 01 as the information 502 of the 
cut point (in the step 602). This cut point of the moving image data 101 shows a point where the scene of the moving 
image data 1 01 changes or a point where the camera for photographing the moving image data 1 01 switches. 
[0235] Next, the extracting unit 503 for extracting the information of the unique identification of the moving image 
arranges at least one value from values of the physical information of the feature quantity of the moving image (such as 
the information of the luminance and the color difference (Y, Cb, Cr) of each frame and / or the information of the color 
(R, G, B) of each frame) based on the information 103, mean value of the values, total value of the values and / or dif- 
ference value of the values on the time axis. Then, the extracting unit 503 picks out all values arranged on the time axis, 
or some values from the values arranged on the time axis at uniform intervals or irregular intervals. And then, the 
extracting unit 503 extracts the picked out values in each shot unit of the moving image data 101 divided by the infor- 
mation 502 of the cut point as the information 505 of the unique identification of the moving image to be used for iden- 
tifying the moving image (in the step 603). 

[0236] The storing unit 106 for storing the information of the unique identification of the moving image stores the 
information 505 of the unique identification of the moving image extracted by the extracting unit 503 (in the step 604). 
[0237] On the other hand, the information 508 of the unique identification of the moving image is extracted from 
other moving image data (the object moving image data for retrieving) (in the step 205). This extracting process can be 
executed by the processing equal to the foregoing steps 601 to 604 (in FIGs.6A and 6B). 

[0238] Then, the retrieving unit 109 for retrieving the information of the unique identification of the moving image 
compares the information 505 of the unique identification of the moving image stored in the storing unit 106 with the 
information 508 of the unique identification of the moving image extracted from the other moving image data (in the step 
206). 

[0239] This comparing result is stored in the storing unit 1 1 0 for storing the retrieving result (in the step 207). In the 
case of displaying and / or printing the comparing result (in the step 208), it is outputted by the output unit 111 for dis- 
playing and / or printing the retrieving result (in the step 209). 

[0240] The process of the above mentioned steps 206 to 209 is executed for all of the information 508 of the unique 
identification of the moving image extracted from the other moving image data (in the step 210). 

[0241] In case the digital moving image data 2601 is the compressed moving image data 301 , the following process 
(the steps 402 to 41 0) is executed. 

[0242] The extracting unit 302 extracts the information 303 of the feature quantity of the compressed moving image, 
which includes the DCT coefficients of each frame and the information of the motion compensation of each frame, from 
the compressed moving image data 301 (in the step 402). And, the detecting unit 701 detects the cut point of the com- 
pressed moving image data 301 as the information 702 of the cut point (in the step 802). 

[0243] Next, the extracting unit 703 for extracting the information of the unique identification of the compressed 
moving image calculates values of physical information of the feature quantity of the compressed moving image from 
the DCT coefficients of each frame, which is included in the information 303, mean value of the values, total value of 
the values and / or difference value of the values. On the other hand, the extracting unit 703 calculates values such as 
a motion vector, a mean motion vector between frames before and behind, a total motion vector, a difference vector and 
/ or a motion vector on a whole frame from the information of the motion compensation. And the extracting unit 703 
arranges at least one value from the above mentioned values on the time axis and then picks out all values arranged 
on the time axis, or some values from the values arranged on the time axis at uniform intervals or irregular intervals. 
And then, the extracting unit 703 extracts the picked out values in each shot unit of the compressed moving image data 
301 divided by using the information 702 of the cut point as the information 705 of the unique identification of the com- 
pressed moving image to be used for identifying the compressed moving image (in the step 803). 
[0244] The storing unit 306 for storing the information of the unique identification of the compressed moving image 
stores the information 705 of the unique identification of the compressed moving image extracted by the extracting unit 
703 (in the step 804). 

[0245] On the other hand, the information 708 of the unique identification of the compressed moving image is 
extracted from other compressed moving image data (the object compressed moving image data for retrieving) (in the 
step 405). This extracting process can be executed by the processing equal to the foregoing steps 801 to 804 (in FIGs. 
8A and 8B). 

[0246] Then, the retrieving unit 309 for retrieving the information of the unique identification of the compressed 
moving image compares the information 705 of the unique identification of the compressed moving image stored in the 
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storing unit 306 with the information 708 of the unique identification of the compressed moving image extracted from 
the other moving image data (in the step 406). 

[0247] This comparing result is stored in the storing unit 11 0 for storing the retrieving result (in the step 207). In the 
case of displaying and / or printing the comparing result (in the step 208), it is outputted by the output unit 111 for dis- 
playing and / or printing the retrieving result (in the step 209). 

[0248] The process of the above mentioned steps 406 to 209 is executed for all of the information 708 of the unique 
identification of the compressed moving image extracted from the other moving image data (in the step 410). 
[0249] As mentioned above, according to the fourteenth embodiment of the invention, whether the inputted digital 
moving image data 2601 is the uncompressed moving image data 101 or the compressed moving image data 301 can 
be judged automatically. Consequently, identical moving image can be retrieved and identified from the moving image 
at the higher speed regardless of the type (compressed / uncompressed) of retrieving image data. 
[0250] An apparatus and a method for retrieving moving image in the fifteenth preferred embodiment according to 
the invention will be explained in FIGs. 30, 31 A and 31 B. 

[0251] FIG. 30 is a block diagram showing an apparatus for retrieving moving image according to the present inven- 
tion. Still, an identical code is appended to the thing having the identical composition in FIG. 30 with FIG. 9. In FIG. 30, 
the apparatus for retrieving moving image comprises an extracting unit 901 for extracting information of motion compen- 
sation in the M x N (M and N are natural numbers) pixels unit of moving image data 1 01 , a DCT (Discrete Cosine Trans- 
form) operation unit 902 for obtaining a DCT coefficient by executing a DCT operation in the M' x N' (M' and N' are 
natural numbers) pixels unit of the moving image data 101 , an extracting unit 302 for extracting information 303 of the 
feature quantity of compressed moving image, which includes DCT coefficients of each frame and information of motion 
compensation of each frame, from compressed moving image data 301 provided by the standard compression encod- 
ing process for moving image such as MPEG, an extracting unit 304 for arranging at least one information of feature 
quantity from the DCT coefficients of each frame and the information of the motion compensation of each frame, which 
are the information 303 extracted by the extracting unit 302, or by the extracting unit 901 and the DCT operation unit 
902, on the time axis and for extracting information 305 of the unique identification of compressed moving image to 
identify the compressed moving image from the arrangement of the information of feature quantity on the time axis, a 
storing unit 306 for storing the information 305 of the unique identification of the compressed moving image extracted 
by the extracting unit 304, a retrieving unit 309 for retrieving identical information with the information 305 of the unique 
identification of the compressed moving image stored in the storing unit 306 from information 308 of the unique identi- 
fication of compressed moving image beforehand extracted from other moving image data, a storing unit 1 10 for storing 
the retrieving result provided by the retrieving unit 309, an output unit 111 for displaying and / or printing the retrieving 
result stored in the storing unit 110, and a judging unit 2602 forjudging whether inputted digital moving image data 2601 
is the moving image data 1 01 or the compressed moving image data 301 and for outputting the moving image data 1 01 
to the extracting unit 102 and the compressed moving image data 301 to the extracting unit. 

[0252] FIGs. 31 A and 31 B are flowcharts of the process in the apparatus for retrieving the moving image according 
to the present invention described in FIG. 30. Still, an identical step number is appended to the step having the identical 
process in FIGs. 31 A and 31 B with FIGs. 1 0A and 1 0B. In FIGs. 30, 31 A and 31 B, at first the compressed / uncom- 
pressed judging unit 2602 acquires the digital moving image data 2601 (in the step 2701). Then, the compressed / 
uncompressed judging unit 2602 judges whether the acquired digital moving image data 2601 is the uncompressed 
moving image data 101 or the compressed moving image data 301 (in the step 2702). 

[0253] In case the digital moving image data 2601 is the uncompressed moving image data 101, the compressed 
/ uncompressed judging unit 2602 outputs the moving image data 1 01 to the extracting unit 901 for extracting the infor- 
mation of the motion compensation and the DCT operation unit 902. And, the following process (the steps 1 003 to 41 0) 
is executed. 

[0254] The extracting unit 901 divides each frame of the moving image data 1 01 into the M x N (M and N are natural 
numbers) pixels units, then calculates the information of the motion compensation in the M x N pixels unit (in the step 
1003). On the other hand, the DCT operation unit 902 divides each frame of the moving image data 101 into the M' x 
N' (M' and N' are natural numbers) pixels units, then calculates the DCT coefficient by executing the DCT operation in 
the M' x N' pixels unit (in the step 1004). 

[0255] On the other hand, in case the digital moving image data 2601 is the compressed moving image data 301 , 
the compressed / uncompressed judging unit 2602 outputs the compressed moving image data 301 to the extracting 
unit 302 for extracting the information of the feature quantity of the compressed moving image. And, the following proc- 
ess (the steps 402 to 410) is executed. 

[0256] The extracting unit 302 extracts the information 303 of the feature quantity of the compressed moving image, 
which includes the DCT coefficients of each frame and the information of the motion compensation of each frame, from 
the compressed moving image data 301 (in the step 402). 

[0257] Next, the extracting unit 304 for extracting the information of the unique identification of the compressed 
moving image calculates values of physical information of the feature quantity of the compressed moving image from 
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the DCT coefficients of each frame, which is included in the information 303 extracted by extracting unit 302, or the 
extracting unit 901 and the DCT operation unit 902, mean value of the values, total value of the values and / or differ- 
ence value of the values. On the other hand, the extracting unit 304 calculates values such as a motion vector, a mean 
motion vector between frames before and behind, a total motion vector, a difference vector and / or a motion vector on 
a whole frame from the information of the motion compensation. And the extracting unit 304 arranges at least one value 
from the above mentioned values on the time axis and then picks out all values arranged on the time axis, or some val- 
ues from the values arranged on the time axis at uniform intervals or irregular intervals. And then, the extracting unit 
304 extracts the picked out values as the information 305 of the unique identification of the compressed moving image 
to be used for identifying the compressed moving image (in the step 403). 

[0258] The storing unit 306 for storing the information of the unique identification of the compressed moving image 
stores the information 305 of the unique identification of the compressed moving image extracted by the extracting unit 
304 (in the step 404). 

[0259] On the other hand, the information 308 of the unique identification of the compressed moving image is 
extracted from other compressed moving image data (the object compressed moving image data for retrieving) (in the 
step 405). This extracting process can be executed by the processing equal to the foregoing steps 1 001 to 404 (in FIGs. 
10A and 10B). 

[0260] Then, the retrieving unit 309 for retrieving the information of the unique identification of the compressed 
moving image compares the information 305 of the unique identification of the compressed moving image stored in the 
storing unit 306 with the information 308 of the unique identification of the compressed moving image extracted from 
the other moving image data (in the step 406). 

[0261] This comparing result is stored in the storing unit 1 1 0 for storing the retrieving result (in the step 207). In the 
case of displaying and / or printing the comparing result (in the step 208), it is outputted by the output unit 111 for dis- 
playing and / or printing the retrieving result (in the step 209). 

[0262] The process of the above mentioned steps 406 to 209 is executed for all of the information 308 of the unique 
identification of the compressed moving image extracted from the other moving image data (in the step 410). 
[0263] As mentioned above, according to the fifteenth embodiment of the invention, whether the inputted digital 
moving image data 2601 is the uncompressed moving image data 101 or the compressed moving image data 301 can 
be judged automatically. Consequently, identical moving image can be retrieved and identified from the moving image 
at the higher speed regardless of the type (compressed / uncompressed) of retrieving image data. 
[0264] An apparatus and a method for retrieving moving image in the sixteenth preferred embodiment according to 
the invention will be explained in FIGs. 32, 33A and 33B. 

[0265] FIG. 32 is a block diagram showing an apparatus for retrieving moving image according to the present inven- 
tion. Still, an identical code is appended to the thing having the identical composition in FIG. 32 with FIG. 1 1. In FIG. 
32, the apparatus for retrieving moving image comprises an extracting unit 901 for extracting information of motion com- 
pensation in the M x N (M and N are natural numbers) pixels unit of moving image data 101, a DCT (Discrete Cosine 
Transform) operation unit 902 for obtaining a DCT coefficient by executing a DCT operation in the M' x N' (IVT and N' are 
natural numbers) pixels unit of the moving image data 1 01 , a detecting unit 501 for detecting a cut point of the moving 
image data 101 as information 502 of a cut point, an extracting unit 302 for extracting information 303 of the feature 
quantity of compressed moving image, which includes DCT coefficients of each frame and information of motion com- 
pensation of each frame, from compressed moving image data 301 provided by the standard compression encoding 
process for moving image such as MPEG, a detecting unit 701 for detecting a cut point of the compressed moving 
image data 301 as information 702 of a cut point, an extracting unit 703 for arranging at least one information of feature 
quantity from the DCT coefficients of each frame and the information of the motion compensation of each frame, which 
are included in the information 303 extracted by the extracting unit 901 and the DCT operation unit 902, or the extracting 
unit 302, on the time axis and for extracting information 705 of the unique identification of compressed moving image 
to identify the compressed moving image from the arrangement of the information of the feature quantity on the time 
axis in each shot unit of the compressed moving image data 301 divided by using the information 702 of the cut point, 
a storing unit 306 for storing the information 705 of the unique identification of the compressed moving image extracted 
by the extracting unit 703, a retrieving unit 309 for retrieving identical information with the information 705 of the unique 
identification of the compressed moving image stored in the storing unit 306 from information 708 of the unique identi- 
fication of compressed moving image beforehand extracted from other moving image data, a storing unit 1 1 0 for storing 
the retrieving result provided by the retrieving unit 309, an output unit 111 for displaying and / or printing the retrieving 
result stored in the storing unit 1 10, and a judging unit 2602 forjudging whether inputted digital moving image data 2601 
is the moving image data 1 01 or the compressed moving image data 301 and for outputting the moving image data 101 
to the extracting unit, the DCT (Discrete Cosine Transform) operation unit 902 and the detecting unit, and the com- 
pressed moving image data 301 to the extracting unit 302 and the detecting unit 701 . 

[0266] FIGs. 33A and 33B are flowcharts of the process in the apparatus for retrieving the moving image according 
to the present invention described in FIG. 32. Still, an identical step number is appended to the step having the identical 
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process in FIGs. 33A and 33B with FIGs. 12A and 12B. In FIGs. 32, 33A and 33B, at first the compressed / uncom- 
pressed judging unit 2602 acquires the digital moving image data 2601 (in the step 2701). Then, the compressed / 
uncompressed judging unit 2602 judges whether the acquired digital moving image data 2601 is the uncompressed 
moving image data 101 or the compressed moving image data 301 (in the step 2702). 

[0267] In case the digital moving image data 2601 is the uncompressed moving image data 101 , the compressed 
/ uncompressed judging unit 2602 outputs the moving image data 1 01 to the extracting unit 901 for extracting the infor- 
mation of the motion compensation, the DCT operation unit 902 and the detecting unit 501 for detecting the cut point. 
And, the following process (the steps 1003 to 410) is executed. 

[0268] The extracting unit 901 divides each frame of the moving image data 1 01 into the M x N (M and N are natural 
numbers) pixels units, then calculates the information of the motion compensation in the M x N pixels unit (in the step 
1003). On the other hand, the DCT operation unit 902 divides each frame of the moving image data 101 into the M' x 
N' (M' and N' are natural numbers) pixels units, then calculates the DCT coefficient by executing the DCT operation in 
the M' x N' pixels unit (in the step 1004). 

[0269] And, the detecting unit 501 detects the cut point of the moving image data 1 01 as the information 502 of the 
cut point (in the step 602). 

[0270] On the other hand, in case the digital moving image data 2601 is the compressed moving image data 301 , 
the compressed / uncompressed judging unit 2602 outputs the compressed moving image data 301 to the extracting 
unit 302 for extracting the information of the feature quantity of the compressed moving image and the detecting unit 
701 for detecting the cut point. And, the following process (the steps 402 to 41 0) is executed. 

[0271 ] The extracting unit 302 extracts the information 303 of the feature quantity of the compressed moving image, 
which includes the DCT coefficients of each frame and the information of the motion compensation of each frame, from 
the compressed moving image data 301 (in the step 402). And, the detecting unit 701 detects the cut point of the com- 
pressed moving image data 301 as the information 702 of the cut point (in the step 802). 

[0272] Next, the extracting unit 703 for extracting the information of the unique identification of the compressed 
moving image calculates values of physical information of the feature quantity of the compressed moving image from 
the DCT coefficients of each frame, which is included in the information 303, mean value of the values, total value of 
the values and / or difference value of the values. On the other hand, the extracting unit 703 calculates values such as 
a motion vector, a mean motion vector between frames before and behind, a total motion vector, a difference vector and 
/ or a motion vector on a whole frame from the information of the motion compensation. And the extracting unit 703 
arranges at least one value from the above mentioned values on the time axis and then picks out all values arranged 
on the time axis, or some values from the values arranged on the time axis at uniform intervals or irregular intervals. 
And then, the extracting unit 703 extracts the picked out values in each shot unit of the moving image data 1 01 divided 
by using the information 502 or in each shot unit of the compressed moving image data 301 divided by using the infor- 
mation 702 as the information 705 of the unique identification of the compressed moving image to be used for identify- 
ing the compressed moving image (in the step 803). 

[0273] The storing unit 306 for storing the information of the unique identification of the compressed moving image 
stores the information 705 of the unique identification of the compressed moving image extracted by the extracting unit 
703 (in the step 804). 

[0274] On the other hand, the information 708 of the unique identification of the compressed moving image is 
extracted from other compressed moving image data (the object compressed moving image data for retrieving) (in the 
step 405). This extracting process can be executed by the processing equal to the foregoing steps 1 001 to 804 (in FIGs. 
12A and 12B). 

[0275] Then, the retrieving unit 309 for retrieving the information of the unique identification of the compressed 
moving image compares the information 705 of the unique identification of the compressed moving image stored in the 
storing unit 306 with the information 708 of the unique identification of the compressed moving image extracted from 
the other moving image data (in the step 406). 

[0276] This comparing result is stored in the storing unit 1 1 0 for storing the retrieving result (in the step 207). In the 
case of displaying and / or printing the comparing result (in the step 208), it is outputted by the output unit 111 for dis- 
playing and / or printing the retrieving result (in the step 209). 

[0277] The process of the above mentioned steps 406 to 209 is executed for all of the information 708 of the unique 
identification of the compressed moving image extracted from the other moving image data (in the step 410). 
[0278] As mentioned above, according to the sixteenth embodiment of the invention, whether the inputted digital 
moving image data 2601 is the uncompressed moving image data 101 or the compressed moving image data 301 can 
be judged automatically Consequently, identical moving image can be retrieved and identified from the moving image 
at the higher speed regardless of the type (compressed / uncompressed) of retrieving image data. 
[0279] An apparatus and a method for retrieving moving image in the seventeenth preferred embodiment according 
to the invention will be explained in FIGs. 34, 35A and 35B. 

[0280] FIG. 34 is a block diagram showing an apparatus for retrieving moving image according to the present inven- 
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tion. Still, an identical code is appended to the thing having the identical composition in FIG. 34 with FIG. 13. In FIG. 
34, the apparatus for retrieving moving image comprises an extracting unit 901 for extracting information of motion com- 
pensation in the MxN(M and N are natural numbers) pixels unit of moving image data 1 01 , an extracting unit 1 301 for 
calculating an average pixel value, such as information of average color (R, G, B), and / or information of average lumi- 
nance and average color difference (Y, Cb, Cr), in the M' x N' (M' and N' are natural numbers) pixels unit, an extracting 
unit 1302 for extracting information 1303 of the feature quantity of compressed moving image, which includes DC 
(Direct Current) components of DCT (Discrete Cosine Transform) coefficients of each frame and information of motion 
compensation of each frame, from compressed moving image data 301 provided by the standard compression encod- 
ing process for moving image such as MPEG, an extracting unit 1304 for arranging at least one information of feature 
quantity from the DC components of the DCT coefficients of each frame and the information of the motion compensa- 
tion of each frame or the average pixel value, which are included in the information 1303 extracted by the extracting unit 
901 and the extracting unit 1301, or the extracting unit 1302, on the time axis and for extracting information 305 of the 
unique identification of compressed moving image to identify the compressed moving image from the arrangement of 
the information of the feature quantity on the time axis, a storing unit 306 for storing the information 305 of the unique 
identification of the compressed moving image extracted by the extracting unit 1304, a retrieving unit 309 for retrieving 
identical information with the information 305 of the unique identification of the compressed moving image stored in the 
storing unit 306 from information 308 of the unique identification of compressed moving image beforehand extracted 
from other moving image data, a storing unit 1 10 for storing the retrieving result provided by the retrieving unit 309, an 
output unit 111 for displaying and / or printing the retrieving result stored in the storing unit 1 1 0, and a judging unit 2602 
forjudging whether inputted digital moving image data 2601 is the moving image data 101 or the compressed moving 
image data 301 and for outputting the moving image data 101 to the extracting unit 901 and the extracting unit 1301, 
and the compressed moving image data 301 to the extracting unit 1302. 

[0281] FIGs. 35A and 35B are flowcharts of the process in the apparatus for retrieving the moving image according 
to the present invention described in FIG. 34. Still, an identical step number is appended to the step having the identical 
process in FIGs. 35A and 35B with FIGs. 15A and 15B. In FIGs. 34, 35A and 35B, at first the compressed / uncom- 
pressed judging unit 2602 acquires the digital moving image data 2601 (in the step 2701). Then, the compressed / 
uncompressed judging unit 2602 judges whether the acquired digital moving image data 2601 is the uncompressed 
moving image data 101 or the compressed moving image data 301 (in the step 2702). 

[0282] In case the digital moving image data 2601 is the uncompressed moving image data 101, the compressed 
/ uncompressed judging unit 2602 outputs the moving image data 1 01 to the extracting unit 901 for extracting the infor- 
mation of the motion compensation and the extracting unit 1301 for extracting the average value of the block unit. And, 
the following process (the steps 1003 to 410) is executed. 

[0283] The extracting unit 901 divides each frame of the moving image data 1 01 into the M x N (M and N are natural 
numbers) pixels units, then calculates the information of the motion compensation in the M x N pixels unit (in the step 
1 003). On the other hand, the extracting unit 1 301 divides each frame of the moving image data 1 01 into the IW x N' (M' 
and N' are natural numbers) pixels units, then calculates the average pixel value in the M' x N' pixels unit (in the step 
1502). 

[0284] On the other hand, in case the digital moving image data 2601 is the compressed moving image data 301, 
the compressed / uncompressed judging unit 2602 outputs the compressed moving image data 301 to the extracting 
unit 1 302 for extracting the information of the feature quantity of the compressed moving image. And, the following proc- 
ess (the steps 1503 to 410) is executed. 

[0285] The extracting unit 1302 extracts the information 1303 of the feature quantity of the compressed moving 
image, which includes the DC components of the DCT coefficients of each frame and the information of the motion 
compensation of each frame, from the compressed moving image data 301 (in the step 1503). 

[0286] Next, the extracting unit 1304 for extracting the information of the unique identification of the compressed 
moving image calculates values of physical information of the feature quantity of compressed moving image from the 
DC components of the DCT coefficients of each frame and / or values of the average pixel value of each frame, which 
are included in the information 1303, mean value of the values, total value of the values and / or difference value of the 
values. On the other hand, the extracting unit 1304 calculates values such as a motion vector, a mean motion vector 
between frames before and behind, a total motion vector, a difference vector and / or a motion vector on a whole frame 
from the information of the motion compensation. And the extracting unit 1304 arranges at least one value from the 
above mentioned values on the time axis and then picks out all values arranged on the time axis, or some values from 
the values arranged on the time axis at uniform intervals or irregular intervals. And then, the extracting unit 1304 
extracts the picked out values as the information 305 of the unique identification of the compressed moving image to be 
used for identifying the compressed moving image (in the step 1504). 

[0287] The storing unit 306 for storing the information of the unique identification of the compressed moving image 
stores the information 305 of the unique identification of the compressed moving image extracted by the extracting unit 
1304 (in the step 404). 
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[0288] On the other hand, the information 308 of the unique identification of the compressed moving image is 
extracted from other compressed moving image data (the object compressed moving image data for retrieving) (in the 
step 405). This extracting process can be executed by the processing equal to the foregoing steps 1 001 to 404 (in FIGs. 
15A and 15B). 

[0289] Then, the retrieving unit 309 for retrieving the information of the unique identification of the compressed 
moving image compares the information 305 of the unique identification of the compressed moving image stored in the 
storing unit 306 with the information 308 of the unique identification of the compressed moving image extracted from 
the other moving image data (in the step 406). 

[0290] This comparing result is stored in the storing unit 11 0 for storing the retrieving result (in the step 207). In the 
case of displaying and / or printing the comparing result (in the step 208), it is outputted by the output unit 111 for dis- 
playing and / or printing the retrieving result (in the step 209). 

[0291] The process of the above mentioned steps 406 to 209 is executed for all of the information 308 of the unique 
identification of the compressed moving image extracted from the other moving image data (in the step 410). 
[0292] As mentioned above, according to the seventeenth embodiment of the invention, whether the inputted digital 
moving image data 2601 is the uncompressed moving image data 101 or the compressed moving image data 301 can 
be judged automatically. Consequently, identical moving image can be retrieved and identified from the moving image 
at the higher speed regardless of the type (compressed / uncompressed) of retrieving image data. 
[0293] An apparatus and a method for retrieving moving image in the eighteenth preferred embodiment according 
to the invention will be explained in FIGs. 36, 37A and 37B. 

[0294] FIG. 36 is a block diagram showing an apparatus for retrieving moving image according to the present inven- 
tion. Still, an identical code is appended to the thing having the identical composition in FIG. 36 with FIG. 16. In FIG. 
36, the apparatus for retrieving moving image comprises an extracting unit 901 for extracting information of motion com- 
pensation in the M x N (M and N are natural numbers) pixels unit of moving image data 1 01 , an extracting unit 1 301 for 
calculating an average pixel value such as information of average color (R, G, B) and / or information of average lumi- 
nance and average color difference (Y, Cb, Cr) in the M' x N' (M' and N' are natural numbers) pixels unit, a detecting unit 
501 for detecting a cut point of the moving image data 1 01 as information 502 of a cut point, an extracting unit 1 302 for 
extracting information 1303 of the feature quantity of compressed moving image, which includes DC (Direct Current) 
components of DCT (Discrete Cosine Transform) coefficients of each frame and information of motion compensation of 
each frame, from compressed moving image data 301 provided by the standard compression encoding process for 
moving image such as MPEG, a detecting unit 701 for detecting a cut point of the compressed moving image data 301 
as information 702 of a cut point, an extracting unit 1603 for arranging at least one information of feature quantity from 
the DC components of the DCT coefficients of each frame and the information of the motion compensation of each 
frame or the average pixel values, which are included in the information 1303 extracted by the extracting unit 901 and 
the extracting unit 1301, or the extracting unit 1302, on the time axis and for extracting information 705 of the unique 
identification of compressed moving image to identify the compressed moving image from the arrangement of the infor- 
mation of the feature quantity on the time axis in each shot unit of the moving image data 1 01 divided by using the infor- 
mation 502 of the cut point or in each shot unit of the compressed moving image data 301 divided by using the 
information 702 of the cut point, a storing unit 306 for storing the information 705 of the unique identification of the com- 
pressed moving image extracted by the extracting unit 1 603, a retrieving unit 309 for retrieving identical information with 
the information 705 of the unique identification of the compressed moving image stored in the storing unit 306 from 
information 708 of the unique identification of compressed moving image beforehand extracted from other moving 
image data, a storing unit 1 1 0 for storing the retrieving result provided by the retrieving unit 309, an output unit 111 for 
displaying and /or printing the retrieving result stored in the storing unit 110, and a judging unit 2602 forjudging whether 
inputted digital moving image data 2601 is the moving image data 101 or the compressed moving image data 301 and 
for outputting the moving image data 1 01 to the extracting unit 901 , the extracting unit 1 301 and the detecting unit 501 , 
and the compressed moving image data 301 to the extracting unit 1302 and the detecting unit 701 . 
[0295] FIGs. 37A and 37B are flowcharts of the process in the apparatus for retrieving the moving image according 
to the present invention described in FIG. 36. Still, an identical step number is appended to the step having the identical 
process in FIGs. 37A and 37B with FIGs. 17A and 17B. In FIGs. 36, 37A and 37B, at first the compressed / uncom- 
pressed judging unit 2602 acquires the digital moving image data 2601 (in the step 2701). Then, the compressed / 
uncompressed judging unit 2602 judges whether the acquired digital moving image data 2601 is the uncompressed 
moving image data 101 or the compressed moving image data 301 (in the step 2702). 

[0296] In case the digital moving image data 2601 is the uncompressed moving image data 101 , the compressed 
/ uncompressed judging unit 2602 outputs the moving image data 1 01 to the extracting unit 901 for extracting the infor- 
mation of the motion compensation, the extracting unit 1301 for extracting the average value of the block unit and the 
detecting unit 501 for detecting the cut point. And, the following process (the steps 1003 to 410) is executed. 
[0297] The extracting unit 901 divides each frame of the moving image data 1 01 into the MxN(M and N are natural 
numbers) pixels units, then calculates the information of the motion compensation in the M x N pixels unit (in the step 
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1003). And, the extracting unit 1301 divides each frame of the moving image data 101 into the M' x N' (M' and N' are 
natural numbers) pixels units, then calculates the average pixel value in the M' x N' pixels unit (in the step 1502). And 
more, the detecting unit 501 detects the cut point of the moving image data 1 01 as the information 502 of the cut point 
(in the step 602). 

[0298] On the other hand, in case the digital moving image data 2601 is the compressed moving image data 301, 
the compressed / uncompressed judging unit 2602 outputs the compressed moving image data 301 to the extracting 
unit 1302 for extracting the information of the feature quantity of the compressed moving image and the detecting unit 
701 for detecting the cut point. And, the following process (the steps 1503 to 410) is executed. 

[0299] The extracting unit 1302 extracts the information 1303 of the feature quantity of the compressed moving 
image, which includes the DC components of the DCT coefficients of each frame and the information of the motion 
compensation of each frame, from the compressed moving image data 301 (in the step 1503). And then, the detecting 
unit 701 detects the cut point of the compressed moving image data 301 as the information of the cut point 702 (in the 
step 802). 

[0300] Next, the extracting unit 1603 for extracting the information of the unique identification of the compressed 
moving image calculates values of physical information of the feature quantity of the compressed moving image from 
the DC components of the DCT coefficients of each frame and / or values of the average pixel value of each frame, 
which are included in the information 1303, mean value of the values, total value of the values and / or difference value 
of the values. On the other hand, the extracting unit 1603 calculates values such as a motion vector, a mean motion 
vector between frames before and behind, a total motion vector, a difference vector and / or a motion vector on a whole 
frame from the information of the motion compensation. And the extracting unit 1603 arranges at least one value from 
the above mentioned values on the time axis and then picks out all values arranged on the time axis, or some values 
from the values arranged on the time axis at uniform intervals or irregular intervals. And then, the extracting unit 1603 
extracts the picked out values in each shot unit of the moving image data 101 divided by using the information 502 of 
the cut point or in each shot unit of the compressed moving image data 301 divided by using the information 702 of the 
cut point as the information 705 of the unique identification of the compressed moving image to be used for identifying 
the compressed moving image (in the step 1702). 

[0301] The storing unit 306 for storing the information of the unique identification of the compressed moving image 
stores the information 705 of the unique identification of the compressed moving image extracted by the extracting unit 
1603 (in the step 804). 

[0302] On the other hand, the information 708 of the unique identification of the compressed moving image is 
extracted from other compressed moving image data (the object compressed moving image data for retrieving) (in the 
step 405). This extracting process can be executed by the processing equal to the foregoing steps 1 001 to 804 (in FIGs. 
17A and 17B). 

[0303] Then, the retrieving unit 309 for retrieving the information of the unique identification of the compressed 
moving image compares the information 705 of the unique identification of the compressed moving image stored in the 
storing unit 306 with the information 708 of the unique identification of the compressed moving image extracted from 
the other moving image data (in the step 406). 

[0304] This comparing result is stored in the storing unit 1 1 0 for storing the retrieving result (in the step 207). In the 
case of displaying and / or printing the comparing result (in the step 208), it is outputted by the output unit 111 for dis- 
playing and / or printing the retrieving result (in the step 209). 

[0305] The process of the above mentioned steps 406 to 209 is executed for all of the information 708 of the unique 
identification of the compressed moving image extracted from the other moving image data (in the step 410). 
[0306] As mentioned above, according to the eighteenth embodiment of the invention, whether the inputted digital 
moving image data 2601 is the uncompressed moving image data 101 or the compressed moving image data 301 can 
be judged automatically. Consequently, identical moving image can be retrieved and identified from the moving image 
at the higher speed regardless of the type (compressed / uncompressed) of retrieving image data. 
[0307] An apparatus and a method for retrieving moving image in the nineteenth preferred embodiment according 
to the invention will be explained in FIGs. 38, 39A and 39B. 

[0308] FIG. 38 is a block diagram showing an apparatus for retrieving moving image according to the present inven- 
tion. Still, an identical code is appended to the thing having the identical composition in FIG. 38 with FIG. 18. In FIG. 
38, the apparatus for retrieving moving image comprises a DCT (Discrete Cosine Transform) operation unit 902 for 
obtaining a DCT coefficient by executing a DCT operation in the MxN (M and N are natural numbers) pixels unit of 
moving image data 101 as information 303 of the feature quantity of compressed moving image, an extracting unit 302 
for extracting information 303 of the feature quantity of compressed moving image, which includes DCT coefficients of 
each frame, from compressed moving image data 301 provided by the standard compression encoding process for 
moving image such as MPEG, an extracting unit 304 for arranging at least one information of feature quantity from the 
DCT coefficients of each frame, which is included in the information 303 extracted by the DCT operation unit 902 or by 
the extracting unit 302, on the time axis and for extracting information 305 of the unique identification of compressed 
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moving image to identify the compressed moving image from the arrangement of the information of feature quantity on 
the time axis, a storing unit 306 for storing the information 305 of the unique identification of the compressed moving 
image extracted by the extracting unit 304, a retrieving unit 309 for retrieving identical information with the information 
305 of the unique identification of the compressed moving image stored in the storing unit 306 from information 308 of 
the unique identification of compressed moving image beforehand extracted from other moving image data, a storing 
unit 110 for storing the retrieving result provided by the retrieving unit 309, an output unit 111 for displaying and / or 
printing the retrieving result stored in the storing unit 110, and a judging unit 2602 forjudging whether inputted digital 
moving image data 2601 is the moving image data 101 or the compressed moving image data 301 and for outputting 
the moving image data 101 to the DCT (Discrete Cosine Transform) operation unit 902 and the compressed moving 
image data 301 to the extracting unit 302. 

[0309] FIGs. 39A and 39B are flowcharts of the process in the apparatus for retrieving the moving image according 
to the present invention described in FIG. 38. Still, an identical step number is appended to the step having the identical 
process in FIGs. 39A and 39B with FIGs. 19A and 19B. In FIGs. 38, 39A and 39B, at first the compressed / uncom- 
pressed judging unit 2602 acquires the digital moving image data 2601 (in the step 2701). Then, the compressed / 
uncompressed judging unit 2602 judges whether the acquired digital moving image data 2601 is the uncompressed 
moving image data 101 or the compressed moving image data 301 (in the step 2702). 

[0310] In case the digital moving image data 2601 is the uncompressed moving image data 101 , the compressed 
/ uncompressed judging unit 2602 outputs the moving image data 101 to the DCT operation unit 902. The following 
process (the steps 1 902 to 41 0) is executed. 

[031 1] The DCT operation unit 902 divides each frame of the moving image data 1 01 into the M x N (M and N are 
natural numbers) pixels units, then calculates the DCT coefficient by executing the DCT operation in the M x N pixels 
unit as the information 303 of the feature quantity of the compressed moving image (in the step 1 902). 
[0312] On the other hand, in case the digital moving image data 2601 is the compressed moving image data 301 , 
the compressed / uncompressed judging unit 2602 outputs the compressed moving image data 301 to the extracting 
unit 302 for extracting the information of the feature quantity of the compressed moving image. The following process 
(the steps 1903 to 410) is executed. 

[0313] The extracting unit 302 extracts the information 303 of the feature quantity of the compressed moving image, 
which includes the DCT coefficients of each frame, from the compressed moving image data 301 as (in the step 1 903). 
[0314] Next, the extracting unit 304 for extracting the information of the unique identification of the compressed 
moving image calculates values of physical information of the feature quantity of the compressed moving image from 
the DCT coefficients of each frame, which is included in the information 303 extracted by the extracting unit 902, mean 
value of the values, total value of the values and / or difference value of the values. And the extracting unit 304 arranges 
at least one value from the above mentioned values on the time axis and then picks out all values arranged on the time 
axis, or some values from the values arranged on the time axis at uniform intervals or irregular intervals. And then, the 
extracting unit 304 extracts the picked out values as the information 305 of the unique identification of the compressed 
moving image to be used for identifying the compressed moving image (in the step 1904). 

[0315] The storing unit 306 for storing the information of the unique identification of the compressed moving image 
stores the information 305 of the unique identification of the compressed moving image extracted by the extracting unit 
304 (in the step 404). 

[0316] On the other hand, the information 308 of the unique identification of the compressed moving image is 
extracted from other compressed moving image data (the object compressed moving image data for retrieving) (in the 
step 405). This extracting process can be executed by the processing equal to the foregoing steps 1 001 to 404 (in FIGs. 
19A and 19B). 

[0317] Then, the retrieving unit 309 for retrieving the information of the unique identification of the compressed 
moving image compares the information 305 of the unique identification of the compressed moving image stored in the 
storing unit 306 with the information 308 of the unique identification of the compressed moving image extracted from 
the other moving image data (in the step 406). 

[0318] This comparing result is stored in the storing unit 1 10 for storing the retrieving result (in the step 207). In the 
case of displaying and / or printing the comparing result (in the step 208), it is outputted by the output unit 111 for dis- 
playing and / or printing the retrieving result (in the step 209). 

[0319] The process of the above mentioned steps 406 to 209 is executed for all of the information 308 of the unique 
identification of the compressed moving image extracted from the other moving image data (in the step 410). 
[0320] As mentioned above, according to the nineteenth embodiment of the invention, whether the inputted digital 
moving image data 2601 is the uncompressed moving image data 101 or the compressed moving image data 301 can 
be judged automatically. Consequently, identical moving image can be retrieved and identified from the moving image 
at the higher speed regardless of the type (compressed / uncompressed) of retrieving image data. 
[0321] An apparatus and a method for retrieving moving image in the twentieth preferred embodiment according to 
the invention will be explained in FIGs. 40, 41 A and 41 B. 
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[0322] FIG. 40 is a block diagram showing an apparatus for retrieving moving image according to the present inven- 
tion. Still, an identical code is appended to the thing having the identical composition in FIG. 40 with FIG. 20. In FIG. 
40, the apparatus for retrieving moving image comprises a DCT (Discrete Cosine Transform) operation unit 902 for 
obtaining a DCT coefficient as information 303 of the feature quantity of compressed moving image by executing a DCT 
operation in the M x N (M and N are natural numbers) pixels unit of moving image data 101, a detecting unit 501 for 
detecting a cut point of the moving image data 1 01 as information 502 of a cut point, an extracting unit 302 for extracting 
information 303 of the feature quantity of compressed moving image, which includes DCT coefficients of each frame, 
from compressed moving image data 301 provided by the standard compression encoding process for moving image 
such as MPEG, a detecting unit 701 for detecting a cut point of the compressed moving image data 301 as information 

702 of a cut point, an extracting unit 703 for arranging at least one information of feature quantity from the DCT coeffi- 
cients of each frame, which is included in the information 303 extracted by the DCT operation unit 902 or the extracting 
unit 302, on the time axis and for extracting information 705 of the unique identification of compressed moving image 
to identify the compressed moving image from the arrangement of the information of the feature quantity on the time 
axis in each shot unit of the compressed moving image data 301 divided by using the information 502 of the cut point 
and / or the information 702 of the cut point, a storing unit 306 for storing the information 705 of the unique identification 
of the compressed moving image extracted by the extracting unit 703, a retrieving unit 309 for retrieving identical infor- 
mation with the information 705 of the unique identification of the compressed moving image stored in the storing unit 
306 from information 708 of the unique identification of compressed moving image beforehand extracted from other 
moving image data, a storing unit 1 10 for storing the retrieving result provided by the retrieving unit 309, an output unit 
111 for displaying and / or printing the retrieving result stored in the storing unit 1 1 0, and a judging unit 2602 for judging 
whether inputted digital moving image data 2601 is the moving image data 1 01 or the compressed moving image data 
301 and for outputting the moving image data 101 to the DCT (Discrete Cosine Transform) operation unit 902 and the 
detecting unit 501 , and the compressed moving image data 301 to the extracting unit 302 and the detecting unit 701 . 
[0323] FIGs. 41 A and 41 B are flowcharts of the process in the apparatus for retrieving the moving image according 
to the present invention described in FIG. 40. Still, an identical step number is appended to the step having the identical 
process in FIGs. 41 A and 41 B with FIGs. 21 A and 21 B. In FIGs. 40, 41 A and 41 B, at first the compressed / uncom- 
pressed judging unit 2602 acquires the digital moving image data 2601 (in the step 2701). Then, the compressed / 
uncompressed judging unit 2602 judges whether the acquired digital moving image data 2601 is the uncompressed 
moving image data 101 or the compressed moving image data 301 (in the step 2702). 

[0324] In case the digital moving image data 2601 is the uncompressed moving image data 101, the compressed 
/ uncompressed judging unit 2602 outputs the moving image data 101 to the DCT operation unit 902 and the detecting 
unit 501 for detecting the cut point. And, the following process (the steps 1902 to 410) is executed. 
[0325] The DCT operation unit 902 divides each frame of the moving image data 101 into the M x N (M and N are 
natural numbers) pixels units, then calculates the DCT coefficient by executing the DCT operation in the M x N pixels 
unit as the information 303 of the feature quantity of the compressed moving image (in the step 1902). On the other 
hand, the detecting unit 501 for detecting the cut point detects the cut point of the moving image data 1 01 as the infor- 
mation 502 of the cut point (in the step 602). 

[0326] On the other hand, in case the digital moving image data 2601 is the compressed moving image data 301, 
the compressed / uncompressed judging unit 2602 outputs the compressed moving image data 301 to the extracting 
unit 302 for extracting the information of the feature quantity of the compressed moving image and the detecting unit 
701 for detecting the cut point. The following process (the steps 1 903 to 41 0) is executed. 

[0327] The extracting unit 302 extracts the information 303 of the feature quantity of the compressed moving image, 
which includes the DCT coefficients of each frame, from the compressed moving image data 301 (in the step 1903). 
And, the detecting unit 701 detects the cut point of the compressed moving image data 301 as the information 702 of 
the cut point (in the step 802). 

[0328] Next, the extracting unit 703 for extracting the information of the unique identification of the compressed 
moving image calculates values of physical information of the feature quantity of the compressed moving image from 
the DCT coefficients of each frame, which is included in the information 303, mean value of the values, total value of 
the values and / or difference value of the values. And the extracting unit 703 arranges at least one value from the above 
mentioned values on the time axis and then picks out all values arranged on the time axis, or some values from the val- 
ues arranged on the time axis at uniform intervals or irregular intervals. And then, the extracting unit 703 extracts the 
picked out values in each shot unit of the moving image data 101 divided by using the information 502 or in each shot 
unit of the compressed moving image data 301 divided by using the information 702 as the information 705 of the 
unique identification of the compressed moving image to be used for identifying the compressed moving image (in the 
step 2102). 

[0329] The storing unit 306 for storing the information of the unique identification of the compressed moving image 
stores the information 705 of the unique identification of the compressed moving image extracted by the extracting unit 

703 (in the step 804). 
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[0330] On the other hand, the information 708 of the unique identification of the compressed moving image is 
extracted from other compressed moving image data (the object compressed moving image data for retrieving) (in the 
step 405). This extracting process can be executed by the processing equal to the foregoing steps 1 001 to 804 (in FIGs. 
21Aand21B). 

[0331] Then, the retrieving unit 309 for retrieving the information of the unique identification of the compressed 
moving image compares the information 705 of the unique identification of the compressed moving image stored in the 
storing unit 306 with the information 708 of the unique identification of the compressed moving image extracted from 
the other moving image data (in the step 406). 

[0332] This comparing result is stored in the storing unit 11 0 for storing the retrieving result (in the step 207). In the 
case of displaying and / or printing the comparing result (in the step 208), it is outputted by the output unit 111 for dis- 
playing and / or printing the retrieving result (in the step 209). 

[0333] The process of the above mentioned steps 406 to 209 is executed for all of the information 708 of the unique 
identification of the compressed moving image extracted from the other moving image data (in the step 410). 
[0334] As mentioned above, according to the twentieth embodiment of the invention, whether the inputted digital 
moving image data 2601 is the uncompressed moving image data 101 or the compressed moving image data 301 can 
be judged automatically. Consequently, identical moving image can be retrieved and identified from the moving image 
at the higher speed regardless of the type (compressed / uncompressed) of retrieving image data. 
[0335] An apparatus and a method for retrieving moving image in the twenty-first preferred embodiment according 
to the invention will be explained in FIGs. 42, 43A and 43B. 

[0336] FIG. 42 is a block diagram showing an apparatus for retrieving moving image according to the present inven- 
tion. Still, an identical code is appended to the thing having the identical composition in FIG. 42 with FIG. 22. In FIG. 
42, the apparatus for retrieving moving image comprises an extracting unit 1301 for calculating an average pixel value, 
such as information of average color (R, G, B), and / or information of average luminance and average color difference 
(Y, Cb, Cr), in the M x N (M and N are natural numbers) pixels unit, an extracting unit 1302 for extracting information 
1303 of the feature quantity of compressed moving image, which includes DC (Direct Current) components of DCT 
(Discrete Cosine Transform) coefficients of each frame, from compressed moving image data 301 provided by the 
standard compression encoding process for moving image such as M PEG, an extracting unit 1 304 for arranging at least 
one information of feature quantity from the DC components of the DCT coefficients of each frame or the average pixel 
value, which is included in the information 1303 extracted by the extracting unit 1301 and / or the extracting unit 1302, 
on the time axis and for extracting information 305 of the unique identification of compressed moving image to identify 
compressed moving image from the arrangement of the information of the feature quantity on the time axis, a storing 
unit 306 for storing the information 305 of the unique identification of the compressed moving image extracted by the 
extracting unit 1304, a retrieving unit 309 for retrieving identical information with the information 305 of the unique iden- 
tification of the compressed moving image stored in the storing unit 306 from information 308 of the unique identification 
of compressed moving image beforehand extracted from other moving image data, a storing unit 110 for storing the 
retrieving result provided by the retrieving unit 309, an output unit 111 for displaying and / or printing the retrieving result 
stored in the storing unit 1 1 0, and a judging unit 2602 for judging whether inputted digital moving image data 2601 is 
the moving image data 101 or the compressed moving image data 301 and for outputting the moving image data 101 
to the extracting unit 1301 and the compressed moving image data 301 to the extracting unit 1302. 
[0337] FIGs. 43A and 43B are flowcharts of the process in the apparatus for retrieving the moving image according 
to the present invention described in FIG. 42. Still, an identical step number is appended to the step having the identical 
process in FIGs. 43A and 43B with FIGs. 23A and 23B. In FIGs. 42, 43A and 43B, at first the compressed / uncom- 
pressed judging unit 2602 acquires the digital moving image data 2601 (in the step 2701). Then, the compressed / 
uncompressed judging unit 2602 judges whether the acquired digital moving image data 2601 is the uncompressed 
moving image data 101 or the compressed moving image data 301 (in the step 2702). 

[0338] In case the digital moving image data 2601 is the uncompressed moving image data 101 , the compressed 
/ uncompressed judging unit 2602 outputs the moving image data 1 01 to the extracting unit 1 301 for extracting the aver- 
age value of the block unit. The following process (the steps 1502 to 410) is executed. 

[0339] The extracting unit 1301 divides each frame of the moving image data 101 into the MxN(M and N are nat- 
ural numbers) pixels units, then calculates the average pixel value in the MxN pixels unit as the information 1303 of 
the feature quantity of the compressed moving image (in the step 1502). 

[0340] On the other hand, in case the digital moving image data 2601 is the compressed moving image data 301 , 
the compressed / uncompressed judging unit 2602 outputs the compressed moving image data 301 to the extracting 
unit 1302 for extracting the information of the feature quantity of the compressed moving image. The following process 
(the steps 1503 to 410) is executed. 

[0341] The extracting unit 1302 extracts the information 1303 of the feature quantity of the compressed moving 
image, which includes the DC components of the DCT coefficients of each frame, from the compressed moving image 
data 301 (in the step 1503). 
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[0342] Next, the extracting unit 1304 for extracting the information of the unique identification of the compressed 
moving image calculates values of physical information of the feature quantity of compressed moving image from the 
DC components of the DCT coefficients of each frame and / or values of the average pixel value of each frame, which 
are included in the information 1303, mean value of the values, total value of the values and / or difference value of the 
values. And the extracting unit 1304 arranges at least one value from the above mentioned values on the time axis and 
then picks out all values arranged on the time axis, or some values from the values arranged on the time axis at uniform 
intervals or irregular intervals. And then, the extracting unit 1304 extracts the picked out values as the information 305 
of the unique identification of the compressed moving image to be used for identifying the compressed moving image 
(in the step 1504). 

[0343] The storing unit 306 for storing the information of the unique identification of the compressed moving image 
stores the information 305 of the unique identification of the compressed moving image extracted by the extracting unit 
1304 (in the step 404). 

[0344] On the other hand, the information 308 of the unique identification of the compressed moving image is 
extracted from other compressed moving image data (the object compressed moving image data for retrieving) (in the 
step 405). This extracting process can be executed by the processing equal to the foregoing steps 1 001 to 404 (in FIGs. 
23A and 23B). 

[0345] Then, the retrieving unit 309 for retrieving the information of the unique identification of the compressed 
moving image compares the information 305 of the unique identification of the compressed moving image stored in the 
storing unit 306 with the information 308 of the unique identification of the compressed moving image extracted from 
the other moving image data (in the step 406). 

[0346] This comparing result is stored in the storing unit 1 1 0 for storing the retrieving result (in the step 207). In the 
case of displaying and / or printing the comparing result (in the step 208), it is outputted by the output unit 111 for dis- 
playing and / or printing the retrieving result (in the step 209). 

[0347] The process of the above mentioned steps 406 to 209 is executed for all of the information 308 of the unique 
identification of the compressed moving image extracted from the other moving image data (in the step 410). 
[0348] As mentioned above, according to the twenty-first embodiment of the invention, whether the inputted digital 
moving image data 2601 is the uncompressed moving image data 101 or the compressed moving image data 301 can 
be judged automatically. Consequently, identical moving image can be retrieved and identified from the moving image 
at the higher speed regardless of the type (compressed / uncompressed) of retrieving image data. 
[0349] An apparatus and a method for retrieving moving image in the twenty-second preferred embodiment accord- 
ing to the invention will be explained in FIGs. 44, 45A and 45B. 

[0350] FIG. 44 is a block diagram showing an apparatus for retrieving moving image according to the present inven- 
tion. Still, an identical code is appended to the thing having the identical composition in FIG. 44 with FIG. 24. In FIG. 
44, the apparatus for retrieving moving image comprises an extracting unit 1301 for calculating an average pixel value 
such as information of average color (R, G, B) and / or information of average luminance and average color difference 
(Y, Cb, Cr) in the M x N (M and N are natural numbers) pixels unit of moving image data 1 01 as information 1303 of the 
feature quantity of compressed moving image, a detecting unit 501 for detecting a cut point of the moving image data 
101 as information 502 of a cut point, an extracting unit 1302 for extracting information 1303 of the feature quantity of 
compressed moving image, which includes DC (Direct Current) components of DCT (Discrete Cosine Transform) coef- 
ficients of each frame, from compressed moving image data 301 provided by the standard compression encoding proc- 
ess for moving image such as MPEG, a detecting unit 701 for detecting a cut point of the compressed moving image 
data 301 as information 702 of a cut point, an extracting unit 1 603 for arranging at least one information of feature quan- 
tity from the DC components of the DCT coefficients of each frame or the average pixel value, which is included in the 
information 1 303 extracted by the extracting unit 1 301 and / or the extracting unit 1 302, on the time axis and for extract- 
ing information 705 of the unique identification of compressed moving image to identify the compressed moving image 
from the arrangement of the information of the feature quantity on the time axis in each shot unit of the moving image 
data 1 01 divided by using the information 502 of the cut point or in each shot unit of the compressed moving image data 
301 divided by using the information 702 of the cut point, a storing unit 306 for storing the information 705 of the unique 
identification of the compressed moving image extracted by the extracting unit 1603, a retrieving unit 309 for retrieving 
identical information with the information 705 of the unique identification of the compressed moving image stored in the 
storing unit 306 from information 708 of the unique identification of compressed moving image beforehand extracted 
from other moving image data, a storing unit 1 10 for storing the retrieving result provided by the retrieving unit 309, an 
output unit 1 1 1 for displaying and / or printing the retrieving result stored in the storing unit 1 1 0, and a judging unit 2602 
forjudging whether inputted digital moving image data 2601 is the moving image data 101 or the compressed moving 
image data 301 and for outputting the moving image data 1 01 to the extracting unit and the detecting unit 501 , and the 
compressed moving image data 301 to the extracting unit 1302 and the detecting unit 701. 

[0351] FIGs. 45A and 45B are flowcharts of the process in the apparatus for retrieving the moving image according 
to the present invention described in FIG. 44. Still, an identical step number is appended to the step having the identical 
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process in FIGs. 45A and 45B with FIGs. 25A and 25B. In FIGs. 44, 45A and 45B, at first the compressed / uncom- 
pressed judging unit 2602 acquires the digital moving image data 2601 (in the step 2701). Then, the compressed / 
uncompressed judging unit 2602 judges whether the acquired digital moving image data 2601 is the uncompressed 
moving image data 101 or the compressed moving image data 301 (in the step 2702). 

[0352] In case the digital moving image data 2601 is the uncompressed moving image data 101, the compressed 
/ uncompressed judging unit 2602 outputs the moving image data 1 01 to the extracting unit 1 301 for extracting the aver- 
age value of the block unit and the detecting unit 501 for detecting the cut point. And, the following process (the steps 
1502 to 410) is executed. 

[0353] The extracting unit 1301 divides each frame of the moving image data 1 01 into the MxN(M and N are nat- 
ural numbers) pixels units, then calculates the average pixel value in the M x N pixels unit (in the step 1502). And, the 
detecting unit 501 detects the cut point of the moving image data 1 01 as the information 502 of the cut point (in the step 
602). 

[0354] On the other hand, in case the digital moving image data 2601 is the compressed moving image data 301 , 
the compressed / uncompressed judging unit 2602 outputs the compressed moving image data 301 to the extracting 
unit 1302 for extracting the information of the feature quantity of the compressed moving image and the detecting unit 
701 for detecting the cut point. And, the following process (the steps 1503 to 410) is executed. 

[0355] The extracting unit 1302 extracts the information 1303 of the feature quantity of the compressed moving 
image, which includes the DC components of the DCT coefficients of each frame, from the compressed moving image 
data 301 (in the step 1 503). And, the detecting unit 701 detects the cut point of the compressed moving image data 301 
as the information 702 of the cut point (in the step 802). 

[0356] Next, the extracting unit 1603 for extracting the information of the unique identification of the compressed 
moving image calculates values of physical information of the feature quantity of compressed moving image from the 
DC components of the DCT coefficients of each frame and / or the average pixel value of each frame, which are 
included in the information 1303, mean value of the values, total value of the values and / or difference value of the val- 
ues. And the extracting unit 1603 arranges at least one value from the above mentioned values on the time axis and 
then picks out all values arranged on the time axis, or some values from the values arranged on the time axis at uniform 
intervals or irregular intervals. And then, the extracting unit 1603 extracts the picked out values in each shot unit of the 
moving image data 101 divided by using the information 502 of the cut point or in each shot unit of the compressed 
moving image data 301 divided by using the information 702 of the cut point as the information 705 of the unique iden- 
tification of the compressed moving image to be used for identifying the compressed moving image (in the step 1702). 
[0357] The storing unit 306 for storing the information of the unique identification of the compressed moving image 
stores the information 705 of the unique identification of the compressed moving image extracted by the extracting unit 
1603 (in the step 804). 

[0358] On the other hand, the information 708 of the unique identification of the compressed moving image is 
extracted from other compressed moving image data (the object compressed moving image data for retrieving) (in the 
step 405). This extracting process can be executed by the processing equal to the foregoing steps 1 001 to 804 (in FIGs. 
25A and 25B). 

[0359] Then, the retrieving unit 309 for retrieving the information of the unique identification of the compressed 
moving image compares the information 705 of the unique identification of the compressed moving image stored in the 
storing unit 306 with the information 708 of the unique identification of the compressed moving image extracted from 
the other moving image data (in the step 406). 

[0360] This comparing result is stored in the storing unit 1 1 0 for storing the retrieving result (in the step 207). In the 
case of displaying and / or printing the comparing result (in the step 208), it is outputted by the output unit 111 for dis- 
playing and / or printing the retrieving result (in the step 209). 

[0361] The process of the above mentioned steps 406 to 209 is executed for all of the information 708 of the unique 
identification of the compressed moving image extracted from the other moving image data (in the step 410). 
[0362] As mentioned above, according to the twenty-second embodiment of the invention, whether the inputted dig- 
ital moving image data 2601 is the uncompressed moving image data 101 or the compressed moving image data 301 
can be judged automatically. Consequently, identical moving image can be retrieved and identified from the moving 
image at the higher speed regardless of the type (compressed / uncompressed) of retrieving image data. 
[0363] An apparatus and a method for retrieving moving image in the twenty-third preferred embodiment according 
to the invention will be explained in FIGs. 46, 47A and 47B. 

[0364] FIG. 46 is a block diagram showing an apparatus for monitoring television commercials, which is a specific 
example of the second embodiment according to the present invention. Still, an identical code is appended to the thing 
having the identical composition in FIG. 46 with FIG. 3. 

[0365] In FIG. 46, the apparatus for monitoring television commercials comprises an extracting unit 302A for 
extracting information (a DC component) 303A of the feature quantity of compressed moving image, which includes the 
only DC (Direct Current) components of DCT (Discrete Cosine Transform) coefficients of each frame, from MPEG-2 
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stream (television broadcasting) 301 A compressed by MPEG-2 process, an extracting unit (a calculating unit for calcu- 
lating average color information) 304A for arranging an average value of the DC components of each frame, which is 
included in the information 303A extracted by the extracting unit 302A, on the time axis and for extracting the arrange- 
ment of the average values on the time axis as information (time series information of average color information) 305A 
of the unique identification of compressed moving image to identify the compressed moving image, a storing unit 306 
for storing the information 305A of the unique identification of the compressed moving image extracted by the extracting 
unit 304A, a retrieving unit 309 for retrieving identical information with the information 305A of the unique identification 
of the compressed moving image stored in the storing unit 306 from information (time series information of average 
color information of a television commercial) 308A of the unique identification of compressed moving image beforehand 
extracted from other moving image data, a storing unit 110 for storing the retrieving result provided by the retrieving unit 
309, and an output unit 111 for displaying and / or printing the retrieving result stored in the storing unit 1 1 0. 
[0366] The MPEG-2 stream (the television broadcasting) 301 A is provided by encoding the moving image of the 
current television broadcasting. 

[0367] On the apparatus for monitoring television commercials according to the present invention constituted like 
the above, an operation will be explained in the following. 

[0368] FIGs. 47A and 47B are flowcharts of the process in the apparatus for monitoring television commercials to 
the present invention described in FIG. 46. Still, an identical step number is appended to the step having the identical 
process in FIGs. 47A and 47B with FIGs. 4A and 4B. In FIGs. 46, 47A and 47B, at first the extracting unit 302A for 
extracting the DC component fetches the MPEG-2 stream (the television broadcasting) 301 A to be a retrieval key for 
identifying the compressed moving image (in the step 401 A). Then, the extracting unit 302A extracts the information 
(the DC component) 303A of the feature quantity of the compressed moving image, which includes the only DC com- 
ponents of the DCT coefficients of each frame, from the MPEG-2 stream 301 A (in the step 402A). 
[0369] Next, the extracting unit (the calculating unit for calculating the average color information) 304A for extracting 
the information of the unique identification of the compressed moving image calculates an average value by using the 
DC components of each frame, which is included in the information 303A. Then, the extracting unit 304A arranges the 
average values on the time axis and extracts the arrangement of the average values on the time axis as the information 
(the time series information of the average color information) 305A of the unique identification of the compressed mov- 
ing image to be used for identifying the compressed moving image (in the step 403A). 

[0370] The storing unit 306 for storing the information of the unique identification of the compressed moving image 
stores the information 305A extracted by the extracting unit 304A (in the step 404). 

[0371] In the twenty-third embodiment, the information (the time series information of the average color information) 
305A of the unique identification of the compressed moving image consists of three kinds of time series information, 
which are information of luminance and color difference (Y, Cb, Cr). By making a graph of each time series information, 
which it makes the horizontal line to be time and makes the vertical line to be a value of the average color information, 
the information 305A is shown by three graphs. 

[0372] On the other hand, the information (the time series information of the average color information of the tele- 
vision commercial) 308A of the unique identification of the compressed moving image is extracted from the MPEG-2 
stream of the television commercial (the object compressed moving image data for retrieving) (in the step 405). This 
extracting process can be executed by the processing equal to the foregoing steps 401 A to 404. 

[0373] The MPEG-2 stream of the television commercial is provided by encoding the current television broadcast- 
ing equal to the above mentioned MPEG-2 stream 301 A. 

[0374] Then, the retrieving unit 309 for retrieving the information of the unique identification of the compressed 
moving image compares the information 305A of the unique identification of the compressed moving image stored in 
the storing unit 306 with the information 308A of the unique identification of the compressed moving image extracted 
from the other moving image data (in the step 406). 

[0375] The apparatus for retrieving the moving image of the twenty-third embodiment is considered that is realized 
using the simple facility as PC (Personal Computer). Consequently, in the present television broadcasting, ghost phe- 
nomenon, in which image is multiply reflected by reflection and multipath of the television radio wave, and color slippage 
and the like state arise. A little effect is also generated in the information 305A extracted using the television broadcast- 
ing. 

[0376] As mentioned above, the information 305A is shown by 3 graphs, and effect is also generated in the graph. 
However, since there is sufficient information of unique identification of moving image to be used for identifying the mov- 
ing image in the graph, it is possible to be difficult to receive the effect by expanding and reducing the graph. 
[0377] This comparing result is stored in the storing unit 1 1 0 for storing the retrieving result (in the step 207). In the 
case of displaying and / or printing the comparing result (in the step 208), it is outputted by the output unit 111 for dis- 
playing and / or printing the retrieving result (in the step 209). 

[0378] The process of the above mentioned steps 406 to 209 is executed for all of the information 308A of the 
unique identification of the compressed moving image extracted from the other moving image data (in the step 410). 
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[0379] The apparatus and the method for retrieving the moving image according to the invention are explained the 
above. 

[0380] The experimental system shown in FIG. 46 was mounted, and the experiment testing the effectiveness of 
this invention was executed. In the following, the result of the experiment is shown. 

[0381] In the experiment for retrieving the television commercials that are 152 in number and 37 in type from the 
television broadcasting of 26 hours 15 minutes which is ground wave television broadcasting, the number of omission 
was 0 and the number of over-detection was 0. 

[0382] In the above mentioned methods and apparatuses for retrieving the moving image, which are referred from 
in the first embodiment to in the twenty-third embodiment, a process for dividing an image into a plurality of frames is 
executed. It is also possible to execute a process for dividing each of the frames into "n" ("n M is a natural number 2 or 
over) regions in addition to the above mentioned process. In this case, the information of the unique identification of the 
moving image and / or the information of the unique identification of the compressed moving image for identifying the 
moving image are extracted respectively from the divided regions, and then, the extracted information is applied to the 
information 105 of the unique identification of the moving image and / or the information 305 and 305A of the unique 
identification of the compressed moving image. And, a retrieving unit 109 for retrieving the information of the unique 
identification of the moving image and / or a retrieving unit 309 for retrieving the information of the unique identification 
of the compressed moving image retrieve the identical information with the information 1 05 and / or the information 305 
and 305A from the information 108 of the unique identification of the moving image and / or the information 308 and 
308A of the unique identification of the compressed moving image beforehand extracted. In this retrieving process, the 
identical information with the information 1 05 and / or the information 305 and 305A may be extracted with correspond- 
ing in all regions of the information 1 05 and / or the information 305 and 305A. Like the above, by retrieving a frame with 
a plurality of information values, the precision of the retrieving process may be further improved. 
[0383] And more, by dividing each of the frames into "n" regions and extracting the information 105 and / or the 
information 305 and 305A from each of the regions, it is possible to shorten time series information included in the infor- 
mation 1 05 and / or the information 305 and 305A. 

[0384] Moreover, when the information 1 05 and / or the information 305 and 305A are extracted from "n" regions of 
each of the frames and a process of extracting moving image is executed by using the information 1 05 and / or the infor- 
mation 305 and 305A of limited regions over "n - m" (1 < m < (n-1)), it is possible to retrieve similar moving image with 
the difference in a part of regions. That is to say, the retrieving process may be executed with corresponding in the part 
of the regions selected as objects for retrieving, consequently, it is possible to retrieve similar moving image. 
[0385] Until now, the methods and the apparatuses for retrieving the moving image, which are referred from in the 
first embodiment to in the twenty-third embodiment, are explained. A program for executing processes of the above 
mentioned methods for retrieving the moving image may be recorded on a storage medium that is read-around by a 
computer system. 

[0386] Still, in the above mentioned embodiments according to the invention, although either the uncompressed 
moving image data or the compressed moving image data was applied as the object moving image data for retrieving, 
both the uncompressed moving image data and the compressed moving image data may be applied as the object mov- 
ing image data for retrieving. 

[0387] And, in the above mentioned embodiments according to the invention, the information 105 or 305 was 
extracted by using over one information from each of the frames. However, the information 1 05 or 305 will be extracted 
by using over two information from each of the frames, consequently, the precision of the retrieving process may be 
improved. 

[0388] In addition, when the information 1 05 or 305 is extracted by using over two information from each frame, the 
retrieving unit 1 09 or 309 may retrieve similar moving image according to comparing the information 1 05 or 305 by using 
a part of the information values from each of the frames. 

[0389] Although the invention has been described in its preferred form with a certain degree of particularity, it is 
understood that the present disclosure of the preferred form has been changed in the details of construction and the 
combination and arrangement of parts may be resorted to without departing from the spirit and the scope of the inven- 
tion as hereinafter claimed. 

Claims 

1. An apparatus for retrieving moving image to retrieve pre-determined moving image from object moving image, said 
apparatus comprising: 

first extracting means for extracting the information of the feature quantity of each frame from said pre-deter- 
mined moving image; 

second extracting means for extracting the information of the unique identification of moving image to identify 
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the moving image in accordance with said information of the feature quantity, extracted by said first extracting 
means; and 

retrieving means for retrieving said pre-determined moving image from said object moving image by comparing 
said information of the unique identification, extracted by said second extracting means, with the information of 
5 the unique identification prepared in accordance with said object moving image. 

2. The apparatus for retrieving the moving image of Claim 1 , wherein: 

said pre-determined moving image is uncompressed moving image data; and 
10 said second extracting means arranges at least one information of the feature quantity, extracted by said first 

extracting means, on the time axis, and extracts the information of the unique identification to identify the mov- 
ing image from the information of the feature quantity arranged on said time axis. 

3. The apparatus for retrieving the moving image of Claim 1 , wherein: 

15 

said first extracting means extracts the DCT (Discrete Cosine Transform) coefficients of each frame as the 
information of the feature quantity from said pre-determined moving image; and 

said second extracting means arranges at least one information of the feature quantity, extracted by said first 
extracting means, on the time axis, and extracts the information of the unique identification to identify the mov- 
20 ing image from the information of the feature quantity arranged on said time axis. 

4. The apparatus for retrieving the moving image of Claim 1 , wherein: 

said first extracting means extracts the DCT (Discrete Cosine Transform) coefficients and the information of 
25 motion compensation of each frame, as the information of the feature quantity, from said pre-determined mov- 

ing image; and 

said second extracting means arranges at least one information of the feature quantity, extracted by said first 
extracting means, on the time axis, and extracts the information of the unique identification to identify the mov- 
ing image from the information of the feature quantity arranged on said time axis. 

30 

5. The apparatus for retrieving the moving image of Claim 1 , wherein: 

said predetermined moving image is uncompressed moving image data; 

said first extracting means obtains the information of motion compensation as a block unit of M x N pixels (M 
35 and N are natural numbers) from said uncompressed moving image data, and the information of pixel value, 

such as the information of average color (R, G, B) or the information of average luminance and color difference 
(Y, Cb, Cr), as a block unit of M' x N' pixels (M' and N' are natural numbers) from said uncompressed moving 
image data, and extracts these information as the information of feature quantity; and 

said second extracting means arranges at least one information of the feature quantity, extracted by said first 
40 extracting means, on the time axis, and extracts the information of the unique identification to identify the mov- 

ing image from the information of the feature quantity arranged on said time axis. 

6. The apparatus for retrieving the moving images of Claim 1, wherein: 

45 said pre-determined moving image is uncompressed moving image data; 

said first extracting means extracts the DCT (Discrete Cosine Transform) coefficients as the information of the 
feature quantity by executing a DCT operation at a block unit of M x N pixels (M and N are natural numbers) 
from said uncompressed moving image data; and 

said second extracting means arranges at least one information of the feature quantity, extracted by said first 
so extracting means, on the time axis, and extracts the information of the unique identification to identify the mov- 

ing image from the information of the feature quantity arranged on said time axis. 

7. The apparatus for retrieving the moving image of Claim 1 , wherein: 

55 said pre-determined moving image is uncompressed moving image data; 

said first extracting means extracts the information of pixel value, such as the information of average color (R, 
G, B) or the information of average luminance and color difference (Y, Cb, Cr), as a block unit of M x N pixels 
(M and N are natural numbers) from said uncompressed moving image data, as the information of feature 
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quantity; and 

said second extracting means arranges at least one information of the feature quantity, extracted by said first 
extracting means, on the time axis, and extracts the information of the unique identification to identify the mov- 
ing image from the information of the feature quantity arranged on said time axis. 

The apparatus for retrieving the moving image of Claim 1 , further comprises: 

detecting means for detecting cut points of the moving image from said pre-determined moving image and 
dividing said pre-determined moving image into a plurality of shots at each of the cut points; 
wherein said second extracting means extracts the information of the unique identification to identify the mov- 
ing image at each of the shots, divided by said detecting means, in accordance with said information of the fea- 
ture quantity of moving image, extracted by said first extracting means. 

The apparatus for retrieving the moving image of Claim 8, further comprises: 

judging means forjudging whether said pre-determined moving image is compressed or uncompressed mov- 
ing image data; 

wherein said first extracting means includes; 

first uncompression extracting means for extracting the information of the feature quantity of each frame from 
the uncompressed moving image data; and 

first compression extracting means for extracting the DCT (Discrete Cosine Transform) coefficients of each 
frame as the information of the feature quantity from the compressed moving image data; 
said detecting means includes; 

uncompression detecting means for detecting cut points of the moving image from the uncompressed moving 
image data and dividing said pre-determined moving image into a plurality of shots at each of the cut points; 
and 

compression detecting means for detecting cut points of the moving image from the compressed moving image 
data and dividing said pre-determined moving image into a plurality of shots at each of the cut points; and 
said second extracting means includes; 

second uncompression extracting means for extracting the information of the unique identification to identify 
the moving image at each of the shots, divided by said uncompression detecting means, in accordance with 
said information of the feature quantity of moving image, extracted by said first uncompression extracting 
means; and 

second compression extracting means for arranging at least one information of the feature quantity, extracted 
by said first compression extracting means, on the time axis at each of the shots, divided by said compression 
detecting means, and extracting the information of the unique identification to identify the moving image from 
the information of the feature quantity arranged on said time axis. 

The apparatus for retrieving the moving image of Claim 1 , further comprises: 

judging means forjudging whether said pre-determined moving image is compressed or uncompressed mov- 
ing image data; 

wherein said first extracting means includes; 

first uncompression extracting means for extracting the information of the feature quantity of each frame from 
the uncompressed moving image data; and 

first compression extracting means for extracting the DCT (Discrete Cosine Transform) coefficients of each 
frame as the information of the feature quantity from the compressed moving image data; and 
said second extracting means includes; 

second uncompression extracting means for arranging at least one information of the feature quantity 
extracted by said first uncompression extracting means, on the time axis, and extracting the information of the 
unique identification to identify the moving image from the information of the feature quantity arranged on said 
time axis; and 

second compression extracting means for arranging at least one information of the feature quantity, extracted 
by said first compression extracting means, on the time axis, and extracting the information of the unique iden- 
tification to identify the moving image from the information of the feature quantity arranged on said time axis. 

The apparatus for retrieving the moving image of Claim 1 0, wherein: 
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said first compression extracting means extracts the DCT (Discrete Cosine Transform) coefficients and the 
information of motion compensation of each frame, as the information of the feature quantity, from the com- 
pressed moving image data. 

5 12. The apparatus for retrieving the moving image of Claim 10, wherein: 

said pre-determined moving image is uncompressed moving image data; 

said first uncompression extracting means extracts the DCT (Discrete Cosine Transform) coefficients of each 
frame as the information of the feature quantity from said uncompressed moving image data; and 
10 said second uncompression extracting means arranges at least one information of the feature quantity, 

extracted by said first uncompression extracting means, on the time axis, and extracts the information of the 
unique identification to identify the moving image from the information of the feature quantity arranged on said 
time axis. 

15 13. The apparatus for retrieving the moving image of Claim 10, wherein: 

said pre-determined moving image is uncompressed moving image data; 

said first uncompression extracting means extracts the DCT (Discrete Cosine Transform) coefficients and the 
information of motion compensation of each frame, as the information of the feature quantity, from said uncom- 
20 pressed moving image data; and 

said second uncompression extracting means arranges at least one information of the feature quantity, 
extracted by said first uncompression extracting means, on the time axis, and extracts the information of the 
unique identification to identify the moving image from the information of the feature quantity arranged on said 
time axis. 

25 

14. The apparatus for retrieving the moving image of Claim 10, wherein: 

said pre-determined moving image is uncompressed moving image data; 

said first uncompression extracting means obtains the information of motion compensation as a block unit of 
30 M x N pixels (M and N are natural numbers) from said uncompressed moving image data, and the information 

of pixel value, such as the information of average color (R, G, B) or the information of average luminance and 
color difference (Y, Cb, Cr), as a block unit of M' x N' pixels (M' and N' are natural numbers) from said uncom- 
pressed moving image data, and extracts these information as the information of feature quantity; and 
said second uncompression extracting means arranges at least one information of the feature quantity, 
35 extracted by said first uncompression extracting means, on the time axis, and extracts the information of the 

unique identification to identify the moving image from the information of the feature quantity arranged on said 
time axis. 

15. The apparatus for retrieving the moving images of Claim 10, wherein: 

40 

said pre-determined moving image is uncompressed moving image data; 

said first uncompression extracting means extracts the DCT (Discrete Cosine Transform) coefficients as the 
information of the feature quantity by executing a DCT operation at a block unit of M x N pixels (M and N are 
natural numbers) from said uncompressed moving image data; and 
45 said second uncompression extracting means arranges at least one information of the feature quantity, 

extracted by said first uncompression extracting means, on the time axis, and extracts the information of the 
unique identification to identify the moving image from the information of the feature quantity arranged on said 
time axis. 

so 16. The apparatus for retrieving the moving image of Claim 10, wherein: 

said pre-determined moving image is uncompressed moving image data; 

said first uncompression extracting means extracts the information of pixel value, such as the information of 
average color (R, G, B) or the information of average luminance and color difference (Y, Cb, Cr), as a block unit 
55 of M x N pixels (M and N are natural numbers) from said uncompressed moving image data, as the information 

of feature quantity; and 

said second uncompression extracting means arranges at least one information of the feature quantity, 
extracted by said first uncompression extracting means, on the time axis, and extracts the information of the 
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unique identification to identify the moving image from the information of the feature quantity arranged on said 
time axis. 

17. The apparatus for retrieving the moving image of Claim 1 , further comprising, 

first storing means for storing said information of the unique identification of the moving image. 

18. The apparatus for retrieving the moving image of Claim 1 7, further comprising, 

second storing means for storing a retrieving result obtained by said retrieving means. 

19. The apparatus for retrieving the moving image of Claim 1 , further comprising, 

displaying means for displaying said retrieving result obtained by said retrieving means. 

20. A method for retrieving moving image to retrieve pre-determined moving image from object moving image, the 
method characterised by the steps of: 

(A) extracting the information of the feature quantity of each frame from said pre-determined moving image; 

(B) extracting the information of the unique identification of moving image to identify the moving image in 
accordance with said information of the feature quantity, extracted by said step (A); and 

(C) retrieving said pre-determined moving image from said object moving image by comparing said information 
of the unique identification, extracted by said step (B), with the information of the unique identification prepared 
in accordance with said object moving image. 

21. The method for retrieving the moving image of Claim 20, wherein: 

said pre-determined moving image is uncompressed moving image data; and 

said step (B) arranges at least one information of the feature quantity, extracted by said step (A), on the time 
axis, and extracts the information of the unique identification to identify the moving image from the information 
of the feature quantity arranged on said time axis. 

22. The method for retrieving the moving image of Claim 20, wherein: 

said step (A) extracts the DCT (Discrete Cosine Transform) coefficients of each frame as the information of the 
feature quantity from said pre-determined moving image; and 

said step (B) arranges at least one information of the feature quantity, extracted by said step (A), on the time 
axis, and extracts the information of the unique identification to identify the moving image from the information 
of the feature quantity arranged on said time axis. 

23. The method for retrieving the moving image of Claim 20, wherein: 

said step (A) extracts the DCT (Discrete Cosine Transform) coefficients and the information of motion compen- 
sation of each frame, as the information of the feature quantity, from said pre-determined moving image; and 
said step (B) arranges at least one information of the feature quantity, extracted by said step (A), on the time 
axis, and extracts the information of the unique identification to identify the moving image from the information 
of the feature quantity arranged on said time axis. 

24. The method for retrieving the moving image of Claim 20, wherein: 

said pre-determined moving image is uncompressed moving image data; 

said step (A) obtains the information of motion compensation as a block unit of M x N pixels (M and N are nat- 
ural numbers) from said uncompressed moving image data, and the information of pixel value, such as the 
information of average color (R, G, B) or the information of average luminance and color difference (Y, Cb, Cr), 
as a block unit of M' x N' pixels (M' and N' are natural numbers) from said uncompressed moving image data, 
and extracts these information as the information of feature quantity; and 

said step (B) arranges at least one information of the feature quantity, extracted by said step (A), on the time 
axis, and extracts the information of the unique identification to identify the moving image from the information 
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of the feature quantity arranged on said time axis. 

25. The method for retrieving the moving images of Claim 20, wherein: 

said pre-determined moving image is uncompressed moving image data; 

said step (A) extracts the DCT (Discrete Cosine Transform) coefficients as the information of the feature quan- 
tity by executing a DCT operation at a block unit of M x N pixels (M and N are natural numbers) from said 
uncompressed moving image data; and 

said step (B) arranges at least one information of the feature quantity, extracted by said step (A), on the time 
axis, and extracts the information of the unique identification to identify the moving image from the information 
of the feature quantity arranged on said time axis. 

26. The method for retrieving the moving image of Claim 20, wherein: 

said pre-determined moving image is uncompressed moving image data; 

said step (A) extracts the information of pixel value, such as the information of average color (R, G, B) or the 
information of average luminance and color difference (Y, Cb, Cr), as a block unit of MxN pixels (M and N are 
natural numbers) from said uncompressed moving image data, as the information of feature quantity; and 
said step (B) arranges at least one information of the feature quantity, extracted by said step (A), on the time 
axis, and extracts the information of the unique identification to identify the moving image from the information 
of the feature quantity arranged on said time axis. 

27. The method for retrieving the moving image of Claim 20, between said step (A) and said step (B), further comprises 
the step of: 

(A1) detecting cut points of the moving image from said pre-determined moving image and dividing said pre- 
determined moving image into a plurality of shots at each of the cut points; 

wherein said step (B) extracts the information of the unique identification to identify the moving image at each 
of the shots, divided by said detecting means, in accordance with said information of the feature quantity of 
moving image, extracted by said step (A). 

28. The method for retrieving the moving image of Claim 27, before said step (A), further comprises the step of: 

(AO) judging whether said pre-determined moving image is compressed or uncompressed moving image data; 
wherein said step (A) includes the steps of; 

(A-1) extracting the information of the feature quantity of each frame from the uncompressed moving image 
data; and 

(A-2) extracting the DCT (Discrete Cosine Transform) coefficients of each frame as the information of the fea- 
ture quantity from the compressed moving image data; 
said step (A1) includes the steps of; 

(A1-1) detecting cut points of the moving image from the uncompressed moving image data and dividing said 
pre-determined moving image into a plurality of shots at each of the cut points; and 

(A1 -2) detecting cut points of the moving image from the compressed moving image data and dividing said pre- 
determined moving image into a plurality of shots at each of the cut points; and 
said step (B) includes the steps of; 

(B-1) extracting the information of the unique identification to identify the moving image at each of the shots, 
divided by said step (A1-1), in accordance with said information of the feature quantity of moving image, 
extracted by said step (A-1); and 

(B-2) arranging at least one information of the feature quantity, extracted by said step (A-2), on the time axis at 
each of the shots, divided by said step (A1 -2), and extracting the information of the unique identification to iden- 
tify the moving image from the information of the feature quantity arranged on said time axis. 

29. The method for retrieving the moving image of Claim 20, before said step (A), further comprises the step of: 

(AO) judging whether said pre-determined moving image is compressed or uncompressed moving image data; 
wherein said step (A) includes the steps of; 

(A-1) extracting the information of the feature quantity of each frame from the uncompressed moving image 
data; and 
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(A-2) extracting the DCT (Discrete Cosine Transform) coefficients of each frame as the information of the fea- 
ture quantity from the compressed moving image data; and 
said step (B) includes the steps of; 

(B-1) arranging at least one information of the feature quantity, extracted by said step (A-1), on the time axis, 
and extracting the information of the unique identification to identify the moving image from the information of 
the feature quantity arranged on said time axis; and 

(B-2) arranging at least one information of the feature quantity, extracted by said step (A-2), on the time axis, 
and extracting the information of the unique identification to identify the moving image from the information of 
the feature quantity arranged on said time axis. 

30. The method for retrieving the moving image of Claim 29, wherein: 

said step (A-2) the DCT (Discrete Cosine Transform) coefficients and the information of motion compensation 
of each frame, as the information of the feature quantity, from the compressed moving image data. 

31. The method for retrieving the moving image of Claim 29, wherein: 

said pre-determined moving image is uncompressed moving image data; 

said step (A-1) extracts the DCT (Discrete Cosine Transform) coefficients of each frame as the information of 
the feature quantity from said uncompressed moving image data; and 

said step (B-1) arranges at least one information of the feature quantity, extracted by said step (A-1), on the 
time axis, and extracts the information of the unique identification to identify the moving image from the infor- 
mation of the feature quantity arranged on said time axis. 

32. The method for retrieving the moving image of Claim 29, wherein: 

said pre-determined moving image is uncompressed moving image data; 

said step (A-1) extracts the DCT (Discrete Cosine Transform) coefficients and the information of motion com- 
pensation of each frame, as the information of the feature quantity, from said uncompressed moving image 
data; and 

said step (B-1) arranges at least one information of the feature quantity, extracted by said step (A-1), on the 
time axis, and extracts the information of the unique identification to identify the moving image from the infor- 
mation of the feature quantity arranged on said time axis. 

33. The method for retrieving the moving image of Claim 29, wherein: 

said pre-determined moving image is uncompressed moving image data; 

said step (A-1) obtains the information of motion compensation as a block unit of M x N pixels (M and N are 
natural numbers) from said uncompressed moving image data, and the information of pixel value, such as the 
information of average color (R, G, B) or the information of average luminance and color difference (Y, Cb, Cr), 
as a block unit of M' x N' pixels (M' and N' are natural numbers) from said uncompressed moving image data, 
and extracts these information as the information of feature quantity; and 

said step (B-1) arranges at least one information of the feature quantity, extracted by said step (A-1), on the 
time axis, and extracts the information of the unique identification to identify the moving image from the infor- 
mation of the feature quantity arranged on said time axis. 

34. The method for retrieving the moving images of Claim 29, wherein: 

said pre-determined moving image is uncompressed moving image data; 

said step (A-1) extracts the DCT (Discrete Cosine Transform) coefficients as the information of the feature 
quantity by executing a DCT operation at a block unit of M x N pixels (M and N are natural numbers) from said 
uncompressed moving image data; and 

said step (B-1) arranges at least one information of the feature quantity, extracted by said step (A-1), on the 
time axis, and extracts the information of the unique identification to identify the moving image from the infor- 
mation of the feature quantity arranged on said time axis. 

35. The method for retrieving the moving image of Claim 29, wherein: 
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said p re-determined moving image is uncompressed moving image data; 

said step (A-1) extracts the information of pixel value, such as the information of average color (R, G, B) or the 
information of average luminance and color difference (Y, Cb, Cr), as a block unit of MxN pixels (M and N are 
natural numbers) from said uncompressed moving image data, as the information of feature quantity; and 
said step (B-1) arranges at least one information of the feature quantity, extracted by said step (A-1), on the 
time axis, and extracts the information of the unique identification to identify the moving image from the infor- 
mation of the feature quantity arranged on said time axis. 

36. The method for retrieving the moving image of Claim 20, between said step (B) and said step (C), further compris- 
ing the step of, 

(B1) storing said information of the unique identification of the moving image. 

37. The method for retrieving the moving image of Claim 38, after said step (C), further comprising the step of, 

(D-1) storing a retrieving result obtained by said retrieving means. 

38. The method for retrieving the moving image of Claim 20, said step (C), further comprising, 

(D-2) displaying said retrieving result obtained by said retrieving means. 

39. A storage medium on which a program is stored, the program for retrieving moving image to retrieve pre-deter- 
mined moving image from object moving image, which is executed by computer system, the program characterised 
by the steps of: 

(A) extracting the information of the feature quantity of each frame from said pre-determined moving image; 

(B) extracting the information of the unique identification of moving image to identify the moving image in 
accordance with said information of the feature quantity, extracted by said step (A); and 

(C) retrieving said pre-determined moving image from said object moving image by comparing said information 
of the unique identification, extracted by said step (B), with the information of the unique identification prepared 
in accordance with said object moving image. 

40. The storage medium on which a program is stored of Claim 39, wherein: 

said pre-determined moving image is uncompressed moving image data; and 

said step (B) arranges at least one information of the feature quantity, extracted by said step (A), on the time 
axis, and extracts the information of the unique identification to identify the moving image from the information 
of the feature quantity arranged on said time axis. 

41. The storage medium on which a program is stored of Claim 39, wherein: 

said step (A) extracts the DCT (Discrete Cosine Transform) coefficients of each frame as the information of the 
feature quantity from said pre-determined moving image; and 

said step (B) arranges at least one information of the feature quantity, extracted by said step (A), on the time 
axis, and extracts the information of the unique identification to identify the moving image from the information 
of the feature quantity arranged on said time axis. 

42. The storage medium on which a program is stored of Claim 39, wherein: 

said step (A) extracts the DCT (Discrete Cosine Transform) coefficients and the information of motion compen- 
sation of each frame, as the information of the feature quantity, from said pre-determined moving image; and 
said step (B) arranges at least one information of the feature quantity, extracted by said step (A), on the time 
axis, and extracts the information of the unique identification to identify the moving image from the information 
of the feature quantity arranged on said time axis. 

43. The storage medium on which a program is stored of Claim 39, wherein: 

said pre-determined moving image is uncompressed moving image data; 
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said step (A) obtains the information of motion compensation as a block unit of M x N pixels (M and N are nat- 
ural numbers) from said uncompressed moving image data, and the information of pixel value, such as the 
information of average color (R, G, B) or the information of average luminance and color difference (Y, Cb, Cr), 
as a block unit of M' x N' pixels (M' and N' are natural numbers) from said uncompressed moving image data, 
and extracts these information as the information of feature quantity; and 

said step (B) arranges at least one information of the feature quantity, extracted by said step (A), on the time 
axis, and extracts the information of the unique identification to identify the moving image from the information 
of the feature quantity arranged on said time axis. 

44. The storage medium on which a program is stored of Claim 39, wherein: 

said pre-determined moving image is uncompressed moving image data; 

said step (A) extracts the DCT (Discrete Cosine Transform) coefficients as the information of the feature quan- 
tity by executing a DCT operation at a block unit of M x N pixels (M and N are natural numbers) from said 
uncompressed moving image data; and 

said step (B) arranges at least one information of the feature quantity, extracted by said step (A), on the time 
axis, and extracts the information of the unique identification to identify the moving image from the information 
of the feature quantity arranged on said time axis. 

45. The storage medium on which a program is stored of Claim 39, wherein: 

said pre-determined moving image is uncompressed moving image data; 

said step (A) extracts the information of pixel value, such as the information of average color (R, G, B) or the 
information of average luminance and color difference (Y, Cb, Cr), as a block unit of M x N pixels (M and N are 
natural numbers) from said uncompressed moving image data, as the information of feature quantity; and 
said step (B) arranges at least one information of the feature quantity, extracted by said step (A), on the time 
axis, and extracts the information of the unique identification to identify the moving image from the information 
of the feature quantity arranged on said time axis. 

46. The storage medium on which a program is stored of Claim 39, between said step (A) and said step (B), further 
comprises the step of: 

(A1) detecting cut points of the moving image from said pre-determined moving image and dividing said pre- 
determined moving image into a plurality of shots at each of the cut points; 

wherein said step (B) extracts the information of the unique identification to identify the moving image at each 
of the shots, divided by said detecting means, in accordance with said information of the feature quantity of 
moving image, extracted by said step (A). 

47. The storage medium on which a program is stored of Claim 46, before said step (A), further comprises the step of: 

(AO) judging whether said pre-determined moving image is compressed or uncompressed moving image data; 
wherein said step (A) includes the steps of; 

(A-1) extracting the information of the feature quantity of each frame from the uncompressed moving image 
data; and 

(A-2) extracting the DCT (Discrete Cosine Transform) coefficients of each frame as the information of the fea- 
ture quantity from the compressed moving image data; 
said step (A1) includes the steps of; 

(A1-1) detecting cut points of the moving image from the uncompressed moving image data and dividing said 
pre-determined moving image into a plurality of shots at each of the cut points; and 

(A1 -2) detecting cut points of the moving image from the compressed moving image data and dividing said pre- 
determined moving image into a plurality of shots at each of the cut points; and 
said step (B) includes the steps of; 

(B-1) extracting the information of the unique identification to identify the moving image at each of the shots, 
divided by said step (A1-1), in accordance with said information of the feature quantity of moving image, 
extracted by said step (A-1); and 

(B-2) arranging at least one information of the feature quantity, extracted by said step (A-2), on the time axis at 
each of the shots, divided by said step (A1 -2), and extracting the information of the unique identification to iden- 
tify the moving image from the information of the feature quantity arranged on said time axis. 



48 



EP 0 996 289 A2 



48. The storage medium on which a program is stored of Claim 39, before said step (A), further comprises the step of: 

(AO) judging whether said pre-determined moving image is compressed or uncompressed moving image data; 
wherein said step (A) includes the steps of; 

(A-1) extracting the information of the feature quantity of each frame from the uncompressed moving image 
data; and 

(A-2) extracting the DCT (Discrete Cosine Transform) coefficients of each frame as the information of the fea- 
ture quantity from the compressed moving image data; and 
said step (B) includes the steps of; 

(B-1) arranging at least one information of the feature quantity, extracted by said step (A-1), on the time axis, 
and extracting the information of the unique identification to identify the moving image from the information of 
the feature quantity arranged on said time axis; and 

(B-2) arranging at least one information of the feature quantity, extracted by said step (A-2), on the time axis, 
and extracting the information of the unique identification to identify the moving image from the information of 
the feature quantity arranged on said time axis. 

49. The storage medium on which a program is stored of Claim 48, wherein: 

said step (A-2) the DCT (Discrete Cosine Transform) coefficients and the information of motion compensation 
of each frame, as the information of the feature quantity, from the compressed moving image data. 

50. The storage medium on which a program is stored of Claim 48, wherein: 

said pre-determined moving image is uncompressed moving image data; 

said step (A-1) extracts the DCT (Discrete Cosine Transform) coefficients of each frame as the information of 
the feature quantity from said uncompressed moving image data; and 

said step (B-1) arranges at least one information of the feature quantity, extracted by said step (A-1), on the 
time axis, and extracts the information of the unique identification to identify the moving image from the infor- 
mation of the feature quantity arranged on said time axis. 

51. The storage medium on which a program is stored of Claim 48, wherein: 

said pre-determined moving image is uncompressed moving image data; 

said step (A-1) extracts the DCT (Discrete Cosine Transform) coefficients and the information of motion com- 
pensation of each frame, as the information of the feature quantity, from said uncompressed moving image 
data; and 

said step (B-1) arranges at least one information of the feature quantity, extracted by said step (A-1), on the 
time axis, and extracts the information of the unique identification to identify the moving image from the infor- 
mation of the feature quantity arranged on said time axis. 

52. The storage medium on which a program is stored of Claim 48, wherein: 

said pre-determined moving image is uncompressed moving image data; 

said step (A-1) obtains the information of motion compensation as a block unit of M x N pixels (M and N are 
natural numbers) from said uncompressed moving image data, and the information of pixel value, such as the 
information of average color (R, G, B) or the information of average luminance and color difference (Y, Cb, Cr), 
as a block unit of M' x N' pixels (IVT and N' are natural information as the information of feature quantity; and 
said step (B-1) arranges at least one information of the feature quantity, extracted by said step (A-1), on the 
time axis, and extracts the information of the unique identification to identify the moving image from the infor- 
mation of the feature quantity arranged on said time axis. 

53. The storage medium on which a program is stored of Claim 48, wherein: 

said pre-determined moving image is uncompressed moving image data; 

said step (A-1) extracts the DCT (Discrete Cosine Transform) coefficients as the information of the feature 
quantity by executing a DCT operation at a block unit of M x N pixels (M and N are natural numbers) from said 
uncompressed moving image data; and 

said step (B-1) arranges at least one information of the feature quantity, extracted by said step (A-1), on the 



49 



EP 0 996 289 A2 



time axis, and extracts the information of the unique identification to identify the moving image from the infor- 
mation of the feature quantity arranged on said time axis. 

54. The storage medium on which a program is stored of Claim 48, wherein: 

said pre-determined moving image is uncompressed moving image data; 

said step (A-1 ) extracts the information of pixel value, such as the information of average color (R, G, B) or the 
information of average luminance and color difference (Y, Cb, Cr), as a block unit of M x N pixels (M and N are 
natural numbers) from said uncompressed moving image data, as the information of feature quantity; and 
said step (B-1) arranges at least one information of the feature quantity, extracted by said step (A-1), on the 
time axis, and extracts the information of the extracted by said step (A-1), on the time axis, and extracts the 
information of the unique identification to identify the moving image from the information of the feature quantity 
arranged on said time axis. 

55. The storage medium on which a program is stored of Claim 39, between said step (B) and said step (C), further 
comprising the step of, 

(B1) storing said information of the unique identification of the moving image. 

56. The storage medium on which a program is stored of Claim 56, after said step (C), further comprising the step of, 

(D-1) storing a retrieving result obtained by said retrieving means. 

57. The storage medium on which a program is stored of Claim 39, said step (C), further comprising, 

(D-2) displaying said retrieving result obtained by said retrieving means. 
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